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Abstract 


The  purpose  of  this  study  is  to  investigate  the  use  of  factor  scores  for  improving 
predictive  validity  in  personnel  selection.  Recent  studies  have  shown  that  general  ability 
is  a  good  predictor  of  future  performance.  However,  specific  factors  may  be  important 
indicators  for  personnel  selection  as  well.  In  the  latent  variable  context,  the  hierarchical 
model  which  considers  general  and  specific  factors  simultaneously  offers  possibilities  for 
solving  prediction  problems  efficiently  ^d  parsimoniously.  This  type  of  modeling  is 
useful  in  determining  the  relative  importance  of  general  and  specific  factors  as  predictors 
of  the  criterion  variable,  and  for  improving  person-job  match.  Two  approaches  are  used 
in  the  current  investigation:  an  artificial  data  is  used  to  study  the  prediction  of  future 
performance,  and  a  real  data  application  using  the  Army  Project  A  and  the  Marine  Corps 
JPM  enlistment  and  performance  data  is  used  to  study  the  practical  implications  of  the 
use  of  specific  factors. 

The  results  of  this  study  show  that  using  specific  factors  in  addition  to  the  general 
factor  as  predictors  provides  better  selection  decisions.  The  illustration  using  the  real 
data  analyses  suggests  that  including  specific  factors  in  predicting  hands-on  performance 
for  most  of  the  jobs  under  consideration  creates  gains  in  terms  of  sensitivity,  specificity, 
and  proportion  of  correct  decisions. 
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CHAPTER  1 
INTRODUCTION 


Test  scores  are  widely  used  as  the  basis  for  personnel  decisions  such  as  selection 
and  placement  of  employees.  For  example,  applicants  for  clerical  positions  must  pass  a 
typing  test;  colleges  require  a  minimum  score  on  the  Scholastic  Aptitude  Test  (SAT); 
government  employees  must  take  Civil  Service  examinations;  and  preschool  children 
must  meet  minimum  standards  on  intelligence  tests  for  admission  to  many  private 
kindergartens  (Allred,  1991).  In  higher  education,  admission  test  scores  are  commonly 
used  in  decision-making  regarding  the  admission  of  students.  In  specialized  training 
programs,  individuals  may  be  admitted  according  to  their  performance  on  related  aptitude 
tests.  These  testing  programs  provide  an  objective  method  of  screening  individuals  to 
find  those  who  ard  best  qualified  for  the  situation  in  question.  Well-known  selection  tests 
in  the  United  States  include  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
for  selection  into  military  jobs  and  the  General  Aptitude  Test  Battery  employment  test  for 
civilian  jobs.  In  addition,  the  SAT,  GRE,  LSAT,  and  GMAT  are  used  in  the  selection 
process  for  entry  into  colleges  and  universities. 

Test  scores  should  communicate  some  sense  of  how  well  a  person  can  do  a 
particular  job  or  what  aspects  of  the  job  a  person  can  do  well.  Instead,  scores  on 
personnel  selection  tests  currently  reveal  something  about  an  examinee’s  relative  standing 
with  reference  to  all  other  examinees,  which  is  useful  for  ranking  applicants  but  is  not 
very  informative  about  how  a  person  at  any  particular  score  level  will  perform  a  given 
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job.  In  order  for  such  test  scores  to  shed  light  on  how  a  person  will  perform  in  a  specific 
position,  such  measures  of  job  competency  would  need  to  be  referenced  to  some  external 
scale  of  job  requirements,  not  merely  to  the  performance  of  other  potential  employees  on 
the  same  test.  The  interpretation  of  a  performance  test  score  refers  to  the  inferences  about 
job  performance  that  can  be  drawn  from  criterion  test  performance.  Insofar  as  a  criterion 
;  measure  is  representative  of  the  work  that  is  required  on  the  job,  some  kind  of  inference 
is  warranted  from  the  test  to  the  job  domain.  Thus,  the  process  of  investing  a 
performance  score  with  meaning  begins  with  a  careful  study  of  the  job  and  involves 
selecting  tasks  for  testing  that  adequately  represent  the  entire  domain  of  the  job’s 
requirements. 

To  evaluate  a  selection  test,  we  need  to  know  whether  or  not  the  test  is  effective. 
The  effectiveness  of  a  testing  program  in  selecting  appropriate  individuals  depends  on 
how  well  the  test  Scores  predict  later  performance  on  an  objective  performance  measure. 
This  is  called  predictive  validity.  Predictive  validity  indicates  the  extent  to  which  an 
individual  s  future  level  on  the  criterion  is  predicted  from  prior  test  performance 
(Messick,  1989).  A  high  degree  of  validity  is  necessary  for  obtaining  a  substantial  benefit 
from  using  the  test  as  a  selection  instrument  (Cronbach  &  Gleser,  1965).  The 
relationship  between  test  scores  and  performance  is  often  expressed  as  a  validity 
coefficient  that  indicates  how  well  the  performance  measure,  or  criterion,  is  predicted  by 
the  test.  In  other  words,  the  measurement  of  performance  should  be  evaluated  before 
adopting  any  tests. 
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The  use  of  factor  scores  as  independent  variables  in  multiple  prediction  has  been 
advocated  in  the  literature  (e.g.,  Morris  &  Guertin,  1977;  Hsu,  1991;  Muthen  &  Hsu, 
1993;  Hsu,  1995).  The  superiority  of  factor  scores  as  predictors  was  affirmed  by  Morris 
and  Guertin  s  finding  that  factor  scores  led  to  a  higher  accuracy  of  prediction  in  a  cross- 
validation  sample  than  did  the  use  of  data-level  variables.  In  addition,  most  latent-trait 
models  have  been  used  with  tests  in  which  performance  is  a  function  of  one  unobserved 
(latent)  characteristic  or  trait,  such  as  vocabulary  level  or  mathematical  reasoning  ability 
(Lord  &  Novick,  1968).  It  is  the  goal  of  such  models  to  estimate  an  examinee’s  standing 
on  the  continuous  latent  trait.  In  particular,  factor  score  computation  takes  into  account 
the  relative  contribution  of  each  variable  to  the  underlying  trait  as  a  function  of  the 
construct  of  a  variable,  as  reflected  in  the  size  of  the  factor  loading  (Lord,  1980;  Short, 
1990;  Muthen,  Kao  &  Burstein,  1991).  In  other  words,  factors  (or  latent  variables)  may 
be  viewed  descriptively  as  representing  the  parsimonious  rank  reduction  for  observation 
obtained  on  a  specific  set  of  subjects  on  a  given  set  of  variables  (Wackwitz  &  Horn, 
1971).  Therefore,  factor  scores  should  approximate  the  true  score  more  closely  than 
scores  obtained  by  compound  measures. 

Two  approaches,  psychometric  and  econometric,  have  different  priorities  for 
evaluating  applicants.  In  personnel  work,  the  traditional,  psychometric,  approach  has 
been  used  to  rank  applicants  in  order  of  their  predicted  performance.  Since  higher  test 
seores  indieate  a  probability  of  better  performance  on  the  job,  the  simple  answer  offered 
by  psychological  measurement  is  that  higher  is  better.  From  this  point  of  view,  job 
performance  is  the  essential  consideration,  and  the  costs  of  increased  performance  are  not 
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calculated.  On  the  other  hand,  because  higher-aptitude  personnel  costs  more  to  recruit, 
management  specialists  tend  to  use  an  econometric  approach  to  questions  of  selection 
standards.  In  a  military  context,  models  for  setting  enlistment  standards  have  recently 
been  designed  to  locate  the  most  cost-effective  cutoff  score  (Zeidner  &  Johnson,  1991a, 
1991b,  1991c).  These  models,  however,  do  not  help  to  answer  the  question  of  how  much 
performance  is  sufficient.  Instead,  they  set  enlistment  standards  in  order  to  minimize 
personnel  costs  per  unit  of  productivity.  From  this  econometric  point  of  view,  cost  is  the 
essential  consideration  (Wigdor  &  Green,  1991).  Neither  of  these  two  approaches,  the 
psychometric  nor  the  econometric,  is  singularly  satisfactory  in  evaluating  applicants.  If 
the  military  is  to  fulfill  its  mission  to  provide  for  the  national  defense,  enlistment 
standards  must  be  set  at  a  level  in  which  to  achieve  the  recruiting  goals  (Hogan  &  Harris, 
1994).  Military  and  civilian  policy-makers  are  interested  in  understanding  the  method  by 
which  qualification  standards  and  recruit  quality  goals  are  established.  Efficient 
analytical  tools  will  provide  a  greater  understanding  of  the  distribution  of  recruit  quality 
needed  to  maintain  adequate  levels  of  job  performance.  Therefore,  the  policy-making 
process  would  benefit  from  an  approach  which  analyze  both  performance  and  cost  (Green 
&  Mavor,  1994). 

General  cognitive  ability  theory  suggests  that  one  general  ability  factor  underlies 
all  specific  cognitive  abilities.  Proponents  of  this  theory  believe  that  the  underlying 
variable,  g,  causes  specific  aptitudes  to  have  validity  in  predicting  job  performance.  If  it 
is  true  that  a  single  factor  (g)  underlies  specific  aptitudes,  and  specific  aptitudes  do  not 
provide  any  greater  prediction  than  g  alone,  then  the  efficient  classification  of  individuals 
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in  jobs  based  on  specific  aptitudes  and  group  aptitudes  is  not  a  pertinent  issue.  However, 
if  there  are  several  factors  that  differentially  predict  performance  in  various  jobs,  then 
classification  efficiency  is  a  relevant  issue.  Specific  aptitude  theory,  on  the  other  hand, 
suggests  that  job  performance  is  best  predicted  by  one  or  more  specific  aptitudes  required 
by  the  job,  rather  than  by  general  cognitive  ability.  For  example,  a  person’s  performance 
as  an  editor  would  be  better  predicted  by  verbal  and  perceptual  speed  abilities  than  by  g 
alone.  According  to  this  theory,  g  has  only  an  indirect  relation  to  job  performance, 
because  it  is  mediated  by  specific  aptitudes.  This  theory  strongly  contributes  to  the 
concept  of  situational  specificity  to  explain  differences  in  Job  requirements  in  different 
settings. 

In  the  latent  variable  context,  the  hierarchical  model  is  one  of  the  important 
cognitive  ability  theories.  Gustafsson  (1988)  explored  the  hierarchical  model  of  abilities 
that  considers  general  and  specific  factors  simultaneously.  He  proposed  a  general  factor, 
g,  that  influences  the  performance  of  examinees  on  all  tests.  In  addition,  he  mentioned  a 
small  number  of  broad  factors  that  influence  groups  of  tests.  Hierarchical  models  of  the 
structure  of  cognitive  abilities  offer  theoretical  as  well  as  practical  advantages 
(Gustafsson,  1988).  Such  models  may  resolve  the  conflict  between  theorists  who 
emphasize  one  general  ability  (e.g.,  Spearman,  Jensen,  and  Humphreys),  and  theorists 
who  emphasize  several  specialized  abilities  (e.g.,  Thurstone,  Guildford,  and  Gardner),  by 
allowing  for  both  categories  of  abilities  in  the  model.  The  hierarchical  approach  also 
offers  possibilities  for  solving  prediction  problems  efficiently  and  parsimoniously 
(Gustafsson,  1988;  Muthen,  1994). 
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Recent  studies  have  shown  general  ability  to  be  a  good  predictor  of  future 
occupational  performance  (Ree  &  Earles,  1991;  Earles  &  Ree,  1992;  Olea  &  Ree,  1994). 
According  to  the  development  of  the  hierarchical  ability  model,  the  contribution  of 
specific  factors  is  an  important  indicator  for  selection  and  classification.  In  the  study  of 
general  and  specific  abilities  as  predictors  of  school  achievement,  Gustafsson  &  Balke 
(1993)  investigated  the  relationship  between  aptitude  variables  and  school  achievement 
using  a  model  of  ability  that  allowed  simultaneous  identification  of  general  and  specific 
abilities.  They  concluded  that  differentiation  among  at  least  a  limited  number  of  broad 
abilities  would  be  worthwhile.  Similar  models  have  been  successfully  applied  to 
studying  subgroup  differences  in  mathematics  achievement  data  (Muthen,  Khoo,  &  Goff, 
1994).  For  most  predictive  tests,  it  is  practical  to  include  both  general  and  specific 
factors  in  the  prediction  model.  For  example,  to  predict  a  person’s  performance  in  a 
special  training  program,  we  would  be  interested  in  knowing  their  ability  on  the  factor 
related  to  that  particular  program  and  less  concerned  about  their  ability  on  other  factors. 
This  type  of  modeling  is  useful  for  finding  the  relative  importance  of  general  and  specific 
factors  as  predictors  of  the  criterion  variable.  Muthen  and  Gustafsson  (1995)  used  this 
approach  to  predict  job  performance.  Their  study  showed  that  different  abilities  were  of 
differential  important  in  different  jobs. 

The  purpose  of  the  current  study  is  to  investigate  the  use  of  factor  scores  for 
improving  predictive  validity  in  personnel  selection  and  to  improve  the  person-job  match. 
Latent  variable  modeling  will  be  used  to  assess  predictive  validity.  Latent  variable 
modeling  is  useful  for  identifying  the  constructs  that  underlie  a  set  of  test  items;  it  makes 
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more  efficient  use  of  the  test  information  than  a  model  which  does  not  use  latent  variables 
(Hsu,  1991;  Muthen  &  Hsu,  1993;  Hsu,  1995),  and  it  is  a  useful  way  of  explicating  the 
validity  of  a  test  in  predicting  future  performance  (Gustafsson,  1988;  Hsu,  1991;  Muthen 
&  Hsu,  1993;  Hsu,  1995;  Muthen  &  Gustafsson,  1995).  This  report  addresses  the 
question:  will  the  approach  of  using  both  general  and  specific  factor  scores  perform  better 
thm  the  method  of  prediction  of  using  the  general  factor  score  only?  From  a  broader 
perspective,  this  report  contributes  to  an  improvement  of  the  person-job  match,  which  is 
in  the  national  interest  given  that  it  will  result  in  better  overall  job  performance  and  more 
effective  utilization  of  the  skills  of  selected  individuals. 

The  current  study  will  use  a  Monte  Carlo  approach  to  assess  the  prediction  of 
future  performance.  In  addition  to  the  Monte  Carlo  study,  a  real  data  application  will  be 
carried  out  to  examine  the  practical  implications  of  the  use  of  factor  scores.  The  Army 
Project  A  and  the  Marine  Corps  Job  Performance  Measurement  (JPM)  enlistment  and 
performance  data  have  been  chosen  as  an  application  of  the  methodology  to  be 
considered. 
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CHAPTER  2 


LITERATURE  REVIEW 

The  literature  review  is  organized  into  four  main  sections.  The  first  section 
addresses  the  issues  of  factor  score  determinacy  and  factor  score  estimation.  The  second 
section  reviews  the  theory  and  practice  in  selection  and  classification.  The  third  section 
looks  at  the  differential  assignment  theory  (Johnson  &  Zeidner,  1991).  The  last  section 
reviews  the  prediction  related  to  the  contribution  of  g  and  5  and  the  concept  of  cross¬ 
validity. 

2. 1  Factor  Score  Determinacy  and  Factor  Score  Estimation 

Beginning  with  Spearman  (1927),  analysts  believed  that  the  factor  score 
indeterminacy  problem  was  resolvable  by  simply  adding  more  observed  variables  in 
order  to  better  define  a  factor.  Cotter  and  Raju  (1982)  showed  that  the  use  of  factor 
scores  in  a  regression  equation  will  significantly  improve  the  prediction  of  population 
squared  cross-validity  as  compared  to  the  straightforward  use  of  data-level  variables. 
Acito  and  Anderson  (1986)  developed  a  simulation  model  to  investigate  the  correlations 
between  the  true  and  derived  common  factor  scores  under  various  data  conditions. 
Indeterminacy  was  found  to  depend  on  the  level  of  communality  and  to  be  detected  more 
accurately  via  image  factoring  than  by  principal  axis  or  principal  component  analysis. 

Because  the  theoretical  scores  are  never  available,  several  systems  for  estimating 
the  factor  scores  have  been  proposed.  As  Tucker  (1971)  noted,  there  are  four  well-known 
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factor  score  estimators,  as  follows:  (1)  Thurstone  proposed  regression  estimates  of  the 
factor  scores,  a  technique  which  has  proven  to  be  very  popular;  (2)  Bartlett  proposed 
estimates  that  minimized  the  sum  of  squared  residuals  weighted  by  the  reciprocals  of  the 
unique  variances  on  the  attributes;  (3)  Horst  characterized  the  least  square  solution;  and 
(4)  Anderson  and  Rubin  proposed  a  variant  of  the  Bartlett  estimates  for  uncorrelated 
factors.  In  addition,  Horn  (1965)  classified  several  techniques  for  computing  factor 
scores,  which  can  be  described  as  (a)  “exact”  procedures  that  are  based  upon  some  form 

r 

of  least  squares  calculation,  and  (b)  “inexact”  procedures  in  which  the  factor  score  is 
estimated  as  a  simple  linear  weighted  sum  of  variable  scores.  Under  the  system 
developed  by  Horn,  a  procedure  is  characterized  by  the  type  of  weighting  factor  that  is 
Used.  The  property  of  the  factor  score  estimators  has  been  examined  with  respect  to 
internal  and  external  characteristics.  Internal  characteristics  reflect  the  covariation  of  the 
estimates  of  other  factors  and  with  the  theoretical  factors.  External  characteristics  reflect 
the  relationship  between  the  factor  scores  and  the  variables  outside  the  factored  battery. 

Despite  the  disagreements  in  the  literature  about  factor  scores,  and  the  availability 
of  several  methods  for  constructing  scores,  few  studies  have  examined  indeterminacy 
directly  by  means  of  empirical  or  simulated  data.  Wackwitz  and  Horn  (1971)  started  with 
“known”  factor  scores  and  generated  “observed  data”  using  a  Monte  Carlo  simulation  to 
compare  principal  component  scores  with  common  factor  scores  derived  from  two 
factoring  methods.  Because  there  are  several  procedures  for  estimating  factor  scores,  the 
question  of  which  factor  score  estimation  procedure  is  better  for  the  purpose  of  prediction 
has  received  a  major  share  of  the  attention  in  studies  conducted  by  Morris  (1979,  1980). 
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Morris’  research  was  based  on  Monte  Carlo  simulation  techniques,  and  there  is  still  a 
great  need  for  studies  comparing  different  factor  score  estimates  using  empirical  data. 
Without  empirical  studies,  the  decision-makers  may  continue  to  fail  to  realize  the  benefits 
of  factor  scores  in  prediction.  The  few  empirical  and  simulated  data  comparisons 
reviewed  leave  many  unanswered  questions.  Empirical  studies  of  analytic  procedures 
using  real  data  have  their  place,  but  their  limitation  is  that  the  ‘‘correct”  answer  is  not 
known.  In  contrast,  simulation  studies  can  start  with  well-defined,  known  parameters, 

thereby  allowing  one  to  assess  the  relative  accuracy  of  techniques  under  different  data 
conditions. 

Short  (1990)  found  that  determinacy  for  the  general  factor  was  robust  in  the 
following  sense:  regardless  of  the  size  of  the  factor  loadings,  the  number  of  items  in  the 
model,  or  the  number  of  items  influenced  by  specific  factors,  the  general  factor 
determinacy  remained  high.  Even  when  specific  factor  loadings  were  higher  than  those 
for  the  general  factor,  the  scores  were  very  reliable.  The  size  of  the  specific  factor 
loading  was  a  major  influence  on  the  specific  factor  determinacy,  as  was  the  size  of  the 
general  factor  loading,  although  to  a  lesser  extent.  The  portion  of  items  influenced  by 
specific  factors  was  found  to  be  more  influential  in  obtaining  reliable  factor  scores  than 
the  number  of  items  overall.  Therefore,  the  specific  factor  scores  were  not  as  reliable  as 
expected.  Short  s  study  also  demonstrated  that  specific  factor  scores  from 
multidimensional,  dichotomized  data  are  not  extremely  reliable.  It  was  suggested  that  the 
initial  test  or  attitude  data  should  be  continuous  and  have  at  least  moderate  factor 
loadings  in  order  to  create  reliable  scores. 
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Research  in  the  area  of  selection  and  predictive  validity  with  latent  variable 
structures  (Hsu,  1991;  Muthen  &  Hsu,  1993;  Hsu,  1995)  found  that  the  factor  score 
estimator  performed  as  well  as  the  theoretical  optimal  method,  which  is  a  full  quasi¬ 
likelihood  estimator  (FQL).  When  regressing  criterion  on  the  estimates  of  factor  scores 
in  a  random  sample,  it  gave  consistent  estimates  of  regression  slopes.  Although  the 
selected  sample  was  not  a  random  sample,  when  selection  was  based  on  the  latent  factors, 
or  on  the  estimated  factor  scores,  the  estimated  regression  slopes  were  still  unbiased 
(Muthen  &  Joreskog,  1983). 

2.2  Selection  and  Classification 

The  general  framework  of  problems  in  prediction  involves  two  elements: 
unknown  criterion  (or  future  performance)  and  known  predictor.  The  desire  is  to  express 
uncertainty  about  criteria  in  the  light  of  predictors.  For  instance,  we  typically  appeal  to  a 
model  that  formalizes  judgment  about  how  predictor  and  criterion  are  related.  If  selection 
was  based  on  the  factor  scores,  the  regression  of  future  performance  on  factor  scores 
would  not  be  distorted  (Muthen  &  Hsu  1993).  A  major  advantage  of  using  factor  scores 
is  because  the  estimated  factor  score  is  a  linear  transformation  of  observed  variables,  the 
factor  score  method  produces  unbiased  estimates  of  slope  when  selection  is  based  on 
observed  variables. 

Most  research  in  prediction  has  a  similar  observed  data  pattern  with  missing  data 
on  criterion  variable  as  shown  in  Figure  1.  This  figure  shows  10  subtests  for  each 
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applicant  (/  -  1  to  N)  while  the  criterion  is  observed  only  for  the  selected  group  (i  =  I  to 
).  For  the  non-selected  group  (/  =  N^+j  to  there  is  no  information  for  the 
criterion.  The  information  from  the  criterion  variables  is  collected  only  for  those  being 
selected.  If  the  selection  procedure  is  based  on  the  observed  predictors,  the  data  can  be 
said  to  be  missing  at  random  (Little  &  Rubin,  1987).  Assuming  missing  at  random, 
maximum-likelihood  estimation  of  latent  variable  models  with  missing  data  can  be 
carried  out  by  structural  equation  modeling  techniques  (Muthen,  Kaplan,  &  Hollis,  1987). 

f 

In  the  latent  variable  approach,  the  criterion  variable  is  regressed  on  the  factors. 
The  fact  that  information  on  the  predictors  is  used  from  both  the  selected  and  non- 
selected  groups  adjusts  for  the  selectivity  in  the  selected  group.  This  approach  is 
analogous  to  the  conventional  Pearson-Lawley  adjustment.  Muthen  and  Hsu  (1993)  and 
Hsu  (1995)  showed  that  factor  score  estimates  perform  about  as  well  as  maximum- 
likelihood  estimates  with  respect  to  estimated  regression  coefficients,  standard  errors, 
standardized  coefficients,  and  R^. 

Research  in  job  classification  has  focused  on  the  appropriate  data  analysis  model 
for  analyzing  the  similarities  and  differences  among  jobs.  In  the  research  performed  by 
Cornelius,  Carron,  and  Collins  (1979),  the  data  analysis  model  was  held  constant  and  the 
type  of  job  analysis  data  was  varied  to  examine  the  effect  on  the  resulting  job 
classification  decisions.  It  is  important  to  realize  that  jobs  can  be  similar  and  dissimilar 
among  different  levels  of  analysis.  Cornelius  et  al.  (1979)  suggested  that  the  selection  of 
the  appropriate  job  analysis  model  is  at  least  as  important  as  the  selection  of  the 
appropriate  data  analysis  model  in  job  classification  research.  Also,  Cornelius,  Schmidt, 
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and  Carron  (1984)  raised  three  different  ways  in  which  job  analysis  is  important  for 
selection  purposes:  (1)  determining  whether  jobs  are  similar  enough  to  be  combined  into 
a  single  selection  system;  (2)  identifying  knowledge,  skills,  and  abilities  (or  aptitudes) 
that  are  important  for  job  performance;  and  (3)  determining  whether  a  test  can  be 
transported  from  a  setting  in  which  it  has  been  shown  to  be  valid  to  a  new  setting. 

Selection  tests  are  used  to  accept  or  reject  an  applicant  for  a  job.  Once  an 
organization  accepts  an  individual  for  employment,  classification  tests  may  then  be  used 
to  assign  an  individual  to  a  specific  training  program  of  a  job  from  among  a  number  of 
available  opportunities.  Meanwhile,  the  purpose  of  classification  is  to  match  individuals 
and  jobs  in  a  manner  that  maximizes  aggregate  performance.  Classification  decisions  are 
a  major  concern  in  military  services  and  are  becoming  an  of  increasing  interest  to 
industry.  Classification  is  also  used  in  counseling  in  order  to  provide  guidance  to 
students  in  the  choice  of  a  field  of  study  or  an  occupation.  Furthermore,  classification  is 
utilized  in  clinical  diagnosis  in  order  to  aid  in  the  choice  of  a  course  of  treatment  (Zeidner 
&  Johnson,  1991b).  Traditionally,  in  selection  and  placement  decisions,  only  a  single  job 
is  involved  and  can  be  accomplished  with  one  or  more  predictors.  The  outcome  of 
selection  and  placement  is  determined  by  an  individual’s  predicted  performance. 
Classification,  however,  requires  multiple  predictors  measuring  more  than  one  dimension 
of  job  performance.  Alley  (1994)  noted  that  the  concept  of  classification  was  broadly 
defined  to  include  selection  as  a  special  case.  If  there  are  multiple  vacancies  for  one  job 
category  and  the  number  of  applicants  is  greater  than  the  number  of  job  vacancies. 
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selection  will  occur.  Classification  usually  implies  multiple  job  categories  and  may  or 
may  not  involve  some  number  of  nonselectees. 

Vineberg  and  Joyner  (1989)  made  the  standard  distinction  between  job 
proficiency  and  job  performance:  namely,  contrasting  what  a  person  knows  or  can  do 
with  what  a  person  actually  does  on  the  job.  Proficiency  usually  is  measured  by  a  paper- 
and-pencil  or  a  hands-on  test  of  job  tasks,  and  is  generally  objective  and  reliable.  Job 
performance  measures,  usually  ratings,  are  generally  subjective  and  less  reliable  than 
proficiency  measures.  Correlation  between  written  job-knowledge  measures  of 
proficiency  and  hands-on  job  sample  measures  of  proficiency  were  generally  found  to  be 
low,  ranging  from  r  =  0.00  to  about  r  =  0.30.  However,  when  job-knowledge  tests  were 
constructed,  based  only  on  information  directly  relevant  to  job  performance,  higher 
correlations  were  found,  ranging  from  r  =  0.58  to  r  =  0.78  (Vineberg  &  Joyner,  1989). 
The  low  reliability  of  ratings  limited  their  relationship  with  other  proficiency  measures, 
with  only  a  few  correlations  appearing  above  r  =  0.30. 

2,3  Differential  Assignment  Theory  (DAT) 

The  underlying  thought  of  the  differential  assignment  theory  (DAT)  approach  is 
based  on  the  concepts  of  classification  efficiency  and  differential  validity  introduced  by 
both  Brogden  and  Horst  (Johnson  &  Zeidner,  1991;  Zeidner  &  Johnson,  1991a,  1991b, 
1991c;  Scholarios,  Johnson,  &  Zeidner,  1994).  The  concept  of  DAT  is  derived  from  an 
integrative  review  of  personnel  classification  literature,  especially  the  contributions  of 
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Brogden  and  Horst,  combined  with  the  systematic  development  of  methodologies  for 
improving  classification  efficiency. 

Zeidner  and  Johnson  (1991a)  proposed  DAT,  postulating  that  several  factors 
differentially  predict  performance  in  various  jobs.  They  believe  that  DAT  provides  a 
more  coherent  framework  for  job  classification,  while  still  recognizing  g  as  the  dominant 
predictor  of  performance.  DAT  stresses  the  difference  between  predicted  performance 
measures  across  jobs  and  explains  classification  efficiency  as  a  function  of  mean 
predictive  validity,  mean  intercorrelation  among  predicted  performance  measures,  the 
number  of  jobs  to  which  individuals  are  assigned,  and  the  selection  ratio.  DAT  states  that 
the  joint  predictor-criterion  space  is  multidimensional  with  useful  factors  contributing  a 
nontrivial  amount  of  classification  efficiency  in  addition  to  the  unidimensional  space 
defined  by  the  g  factor  (Zeidner  &  Johnson,  1991a). 

There  seems  to  be  ample  value  in  tests  of  general  mental  ability  and  tests  of 
specific  abilities.  The  g  proponents  argue  that  the  best  way  to  classify  large  numbers  of 
applicants  in  terms  of  probable  success  is  with  a  measure  of  general  intelligence.  Other 
research  also  suggests  a  continued  interest  in  ability  measurement  beyond  an  exclusive 
psychometric  g  approach.  There  is  good  reason  to  believe  that  the  relevant  issue  is  not 
whether  psychometric  g  or  a  measure  of  a  specific  ability  is  a  better  predictor  of  job 
success,  but  rather  what  are  the  limiting  conditions  to  the  use  of  either  approach.  Lehman 
(1994)  has  applied  the  more  orthodox  cognitive  science  paradigm  to  the  skilled 
performance  problem  and  has  come  to  the  same  conclusion  with  respect  to  the  value  of  a 
multi-ability  view  of  the  world.  The  classic  model  says  simply  that  the  general  factor 
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will  account  for  almost  all  the  relevant  true  score  covariances  among  observed  measures. 
The  goal  of  measurement  is  to  obtain  the  best  possible  measure  of  the  general  factor.  The 
multiple-factor  model  assumes  that  performance  is  multidimensional  and  is  composed  of 
a  number  of  basic  distinguishable  components,  which  are  such  that  some  people  could 
perform  well  on  one  component  but  not  as  well  on  others. 

Scholarios,  Johnson,  and  Zeidner  (1994)  illustrated  that  “Selection  into 'the 
organization  is  first  accomplished  with  a  single  composite  resembling  a  measure  of 
general  cognitive  ability  (g);  assignment  is  then  made  to  specific  jobs  with  weighted  test 
composites  tailored  for  each  job”  (p.  412).  Classification  efficiency  was  measured  as 
mean  predicted  performance  determined  after  optimal  assignment  to  jobs.  Their  stody 
provided  a  comparison  of  differential  assignment  theory  with  general  aptitude  theory  and 
validity  generalization.  The  results  provided  evidence  that  efficient  classification,  using 
tailored  (best  weighted)  test  composites  to  optimally  assign  new  soldiers  to  a  set  of  jobs, 
is  best  accomplished  by  the  design  of  a  test  battery  of  multidimensionality  in  the 
predicted  performance  space.  The  theoretical  value  and  practical  usefulness  of  DAT  was 
supported  by  the  finding  that  both  longer  test  batteries  and  the  use  of  Horst’s  differential 
validity  index  to  select  tests  increase  potential  classification  efficiency. 

2.4  Predictive  Validity  and  Cross-Validity 
2.4.1  Predictive  Validity  in  Relation  to  g  and  s 

In  previous  studies  (Ree  &  Earles,  1991,  1992;  Ree,  Earkesm,  &  Teachout,  1994), 
when  general  {g)  and  specific  (5)  abilities  were  used  to  predict  training  grades  and  job 
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performance,  it  was  found  that  g  was  the  most  potent  predictor  and  that  s  added  little  to 
prediction.  Ree,  Earkesm,  &  Teachouts’  study  (1994)  extended  the  finding  of  statistically 
significant  but  practically  small  incremental  validity  for  specific  measures  to  seven 
additional  jobs  and  to  new  criteria.  It  also  showed  that  the  incremental  value  of  the 
specific  measures  was  small  for  all  three  criteria  and  demonstrated  the  application  of 
estimates  of  effective  sample  size  in  the  computation  of  adjusted  multiple  correlation 
coefficients.  The  average  increment  to  g  by  measures  of  s  was  0.21,  about  the  same  as 
was  found  in  previous  studies  for  both  training  criteria  (Ree  &  Earles,  1991)  and  job 
performance  criteria  (McHenry  et  ah,  1990).  It  is  also  consistent  with  the  estimate 
provided  by  Hunter  and  Hunter  (1984).  Carey  et  al.  (1994)  studied  the  predictive 
efficiency  of  adding  new  tests  to  a  highly  g-saturated  test  battery  for  the  prediction  of 
both  job  performance  and  training  criteria;  he  found  increments  averaging  0.02  across 
these  criteria.  Morales  and  Ree  (1992)  found  similar  incremental  differences  for 
predicting  pilot  and  navigator  performance  that  included  work  sample  criteria.  The 
results  showed  that  g  was  the  best  predictor  of  pilot  and  navigator  job  performance  in  a 
study  of  5,500  airmen.  The  average  validity  ofg  was  0.33  and  the  average  increase  from 
non-g  was  0.05. 

Prediger  (1989)  challenged  the  conclusion  (Hunter  1986;  Jensen  1986;  Thorndike 
1986)  that  general  ability  is  more  important  in  determining  occupational  level  and  job 
performance  than  specific  abilities,  but  he  presented  no  data  on  validity  or  incremental 
validity.  His  conclusion  that  specific  aptitudes  are  important  in  performance  was  based 
on  data  showing  distinct  patterns  of  specific-aptitude  means  across  occupations,  both 
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among  incumbents  and  among  high  school  students  who  later  entered  specific 
occupations.  However,  he  stated  that  validities  can  be  equal  for  different  occupations 
when  means  are  different. 

Hunter  (1985)  reviewed  meta-analyses  of  hundreds  of  studies  showing  that 
general  cognitive  ability  predicts  job  performance  in  all  jobs,  whether  performance  is 
measured  objectively  or  subjectively.  He  also  reviewed  path  analytic  research  consistent 
with  the  theory  that  g  affects  job  performance  primarily  by  improving  job  knowledge,  but 
that  general  ability  also  affects  job  performance  above  and  beyond  its  impact  on  job 
knowledge.  Hunter  discussed  evidence  that,  except  in  a  few  special  cases,  tailoring 
aptitude  composites  to  match  the  job  does  not  improve  the  prediction  of  job  performance 
above  and  beyond  that  provided  by  general  cognitive  ability. 

In  the  Army  Project  A,  three  types  of  measurement  were  used:  (1)  hands-on  tests 
of  job  performance,  (2)  multiple-choice  tests  of  job  knowledge,  and  (3)  ratings  of  job 
performance.  Project  A  was  designed  to  focus  on  individual  differences  in  predictors  and 
on  performance  measures,  and  to  evaluate  the  relationship  between  predictors  and  criteria 
for  a  wide  variety  of  very  different  individuals  (Green  &  Wing,  1988).  The  Army 
researchers  found  that  each  job  was  composed  of  both  elements  unique  to  that  job  and 
elements  shared  by  all  jobs  in  the  Army.  The  findings  also  showed  that  all  predictors 
were  not  equally  valid  for  the  different  aspects  of  job  performance.  Wise  &  McHenry 
(1990)  also  concluded  that  job  performance  is  multidimensional.  Their  major  findings 
were  that  different  predictor  equations  were  needed  for  each  of  the  five  criterion  factors. 


18 


In  addition,  different  prediction  equations  were  required  for  the  component  that  reflected 
proficiency  in  the  technical  tasks  specific  to  each  job. 

2.4.2  Cross-Validity 

The  predictive  power  of  a  sample  regression  equation  in  the  population  and  in 
future  samples  is  often  of  primary  importance  to  researchers.  A  measure  widely  used  for 
this  purpose  is  the  squared  cross-validity  coefficient,  R^.^.  This  index  is -defined  as  the 
squared  correlation  of  actual  criterion  values  with  those  predicted  from  the  sample 
equation  for  the  population  of  interest.  A  natural  choice  as  an  estimator  of  this  parameter 
is  the  sample  squared  multiple  correlation  (Kennedy,  1988). 

Most  authors  suggest  splitting  sample  data,  then  using  one  portion  for 
identification  of  the  model  and  the  other  portion  for  estimation  of  parameters.  But  cross- 
validation  is  known  to  have  significant  restrictions.  In  particular,  a  significant  loss  of 
information  can  be  expected  when  all  available  data  are  not  used  for  purpose  of  parameter 
estimation.  When  sample  size  is  large,  this  loss  in  most  likely  minor.  But  for  a  moderate 
size  dataset,  splitting  data  can  yield  seriously  unstable  parameter  estimates.  Previous 
research  has  shown  that  sample  size  and  the  ratio  of  predictors  selected  to  the  total  in  the 
set  will  affect  validity  estimation  in  the  subset  context. 

Morris  and  Guertin  (1977)  showed  the  superiority  of  factor  scores  as  measured  by 
the  cross-validity  correlations.  They  compared  common  factor  scores  to  unfactored  data- 
level  variables  as  predictors  in  a  regression  equation.  The  Monte  Carlo  study  by  Morris 
and  Guertin  showed  that  the  regression  equations  using  factor  scores  resulted  in  less 
shrinkage  as  a  result  of  cross-validation  than  the  data-level  variables  in  all  cases. 
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Cotter  and  Raju  (1982)  conducted  a  study  to  evaluate  formula-based  population 
squared  cross-validity  and  estimates  of  factor  scores  in  prediction.  They  concluded  that 
formula-based  estimates  of  population  squared  cross-validity  are  as  good  as  those 
obtained  from  the  conventional  cross-validation  procedure. 

There  are  four  correlations  that  are  important  in  a  validation  study  involving 
several  predictors:  the  sample  square  multiple  correlation  (r^),  the  sample  squared  cross¬ 
validity  correlation  (r^^),  the  population  squared  multiple  correlation  (p^),  and  the 
population  squared  cross-validity  correlation  (p/).  The  population  squared  multiple 
correlation  is  the  square  of  the  multiple  correlation  developed  on  the  entire  population. 
The  population  squared  cross-validity  is  based  on  the  regression  weights  developed  on  a 
sample  applied  to  the  entire  population.  The  most  important  correlation  in  selection  is  the 
population  squared  cross-validity,  since  it  provides  a  measure  of  how  well  a  regression 
equation  developed  on  a  sample  will  do  in  future  sampling  from  the  population. 
However,  in  most  situations,  the  population  of  interest  is  not  available  and  the  population 
squared  cross-validity  correlation  cannot  be  calculated  directly.  Consequently,  the 
sample  squared  cross-validity  is  viewed  as  an  estimate  of  population  squared  cross¬ 
validity.  The  cross-validation  procedure  does  have  a  serious  drawback  in  practice.  If  the 
original  sample  is  small,  the  splitting  of  the  sample  into  two  subsamples  of  even  smaller 
size  is  known  to  effect  the  stability  of  the  regression  equation,  thus  raising  questions 
about  the  practicality  of  the  cross-validation  procedure.  Cotter  and  Rajus’  study  (1982) 
suggested  that  the  formula-based  estimation  of  population  squared  cross-validity  is 
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satisfactory,  and  there  is  no  advantage  in  conducting  a  separate,  expensive,  and  time 
consuming  cross-validation  study. 
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CHAPTER  3 


METHODOLOGY 

The  present  study  aims  to  investigate  whether  the  approach  of  prediction  by  using 
specific  factors  in  addition  to  the  general  factor  will  perform  better  than  the  method  of 
prediction  of  using  the  general  factor  score  only.  Technical  definitions  of  predictive 
validity  and  factor  score  determinacy  in  the  context  of  a  latent  variable  model  and  the 
definition  of  sensitivity,  specificity,  and  proportion  of  correct  decisions  in  the  context  of 
the  decision  table  are  discussed  and  defined.  In  the  next  section,  the  Spearman  rank-order 
correlation  coefficient  and  the  cross-validation  are  described  as  indicators  of  the  stability 
of  the  prediction  used  in  the  selection  and  classification.  The  final  section  describes  the 
design  of  the  Monte  Carlo  study  used  to  explore  the  use  of  factor  scores  for  prediction. 

3. 1  Technical  Definitions 

3.1.1  Measurement  Model 

The  measurement  model  (factor  analysis)  is  written  as 

X  =  v  +  At)  +  8, 

where  x  contains  the  predictor  variable  and  s  is  the  residual,  with  covariance  matrix  0. 
The  matrix  A  consists  of  factor  loadings  and  the  vector  t)  consists  of  factor  variables.  We 
have 

E(s)  =  0,  Var(8)  =  0  diagonal. 
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We  assume 


E(r|)  =  a,  Var(Ti)  =  'F. 

Then 

Cov(x)  =  I,y,  =  AT'A'  +  ©. 

Note  that  the  off-diagonal  elements  of  are  functions  of  A  and  they  do  not  involve 

0.  : 

For  identification  we  may  require 

This  model  is  known  as  the  orthogonal  factor  analysis  model.  Then  A'FA'  =  AA  ,  which 
leaves  the  indeterminacy  of  the  multiplication  of  A  on  the  right  by  an  arbitrary  orthogonal 
matrix. 

3.1.2  Predictive  Validity 

Consider  next  the  prediction  equation, 

y  =  a  +  p'r|  +  5, 

where  the  criterion  variable  y  is  regressed  on  the  factor  scores  (t)).  p  is  the  regression 
slope  of  y  on  q.  The  factor  scores  are  estimated  by  using  the  information  on  the  predictor 
variables  (x)  from  the  applicant  sample.  It  is  known  that  for  a  random  sample,  regressing 
y  on  the  regression  estimates  of  factor  scores  gives  consistent  estimates  of  regression 
slopes  (Tucker,  1971;  Hsu,  1991;  Muthen  &  Hsu,  1993;  Hsu,  1995).  Even  if  the  sample 
is  not  a  random  sample,  when  selection  is  based  on  the  estimated  factor  scores,  the 
regression  slope  of  y  on  the  factor  scores  is  still  unbiased.  In  this  report,  predictive 
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validity  is  defined  as  the  regression  coefficients  of  the  criterion  variable  y  on  the  latent 
factor  ri- 

3. 1.3  Resression  Method  of  Factor  Score  Estimation 

Factor  scores  may  be  estimated  for  each  latent  variable  in  the  factor  analysis 
framework.  For  continuous  variable  ;c,  the  standard  regression  method  gives  the 
estimates  ; 

ri=TA'(A'FA'  +0)'';c 

when  both  observed  and  latent  variables  are  standardized  to  zero  mean  (Harris,  1967; 
Maxwell,  1971;  Tucker,  1971;  Muthen,  1978). 

The  procedures  for  the  regression  method  of  factor  score  estimation  are:  (1) 
confirmatory  factor  analysis  on  the  applicant  sample  by  the  LISREL8  program  in  order  to 
determine  the  factor  loadings  (A)  and  the  variances  of  factors  ('F);  (2)  the  regression 
method  for  computing  the  estimated  factor  scores  by  SAS  IML  using  the  factor  model 
estimates;  and  (3)  regression  of  the  criterion  variable  on  the  estimated  factor  scores  for 
the  selected  sample. 

3.1.4  Factor  Score  Determinacv 

Because  the  theoretical  factor  scores  are  never  available,  it  is  important  to 
investigate  the  correlations  between  the  true  and  the  estimated  factor  scores.  These 
values  are  the  factor  score  determinacies.  When  the  r|s  are  standardized,  'F  is  equal  to  I. 
In  this  case,  deteiminacy  may  be  seen  to  depend  on  the  matrix  of  factor  loadings  (A), 
E{ri,T])  =  Q  =  A'  [  AA'  +  ( I  -  diag  ( AA' )]  A. 
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The  correlations  between  the  true  score  (r|)  and  the  estimated  factor  scores  {?})  are  the 

square  roots  of  the  diagonal  elements  of  Q.  (Maxwell,  1971;  Short,  1990). 

3.1.5  Sensitivity,  Specificity,  and  Proportion  of  Correct  Decisions 

A  sensitivity  analysis  is  used  to  evaluate  how  the  prediction  (or  selection 

decision)  is  affected  by  using  the  general  factor  score  only  to  predict  the  criterion 

compared  to  using  the  specific  factor  scores  in  addition  to  the  general  factor  score.  Allred 

(1991)  illustrated  that  a  2  x  2  decision  table  provides  a  method  for  evaluating  the  cost  or 
. 

utility  of  a  test  score  cutoff.  Once  the  selection  ratio  or  the  cutoff  has  been  determined,  a 
simple  2x2  table  can  be  used  to  display  the  number  of  successes  and  failures  in  the 
selected  and  rejected  groups.  The  general  form  of  the  2x2  table  is  shown  in  Figure  2. 
With  this  method,  it  is  simple  to  determine  the  number  of  correct  decisions  made  about 
individuals.  Accepting  an  individual  who  succeeds  and  rejecting  an  individual  who 
would  fail  are  correct  decisions.  The  proportion  of  correct  decisions  is  the  total  of  correct 
accept  (TP)  and  correct  reject  (TN)  divided  by  the  total  number  of  individuals  (N).  In 
Figure  2,  three  equations  for  computing  sensitivity,  specificity,  and  proportion  of  correct 
decisions  are  shown.  Sensitivity  is  defined  as  the  proportion  of  successful  individuals 
who  are  accepted,  while  specificity  is  the  proportion  of  failing  individuals  who  are 
rejected. 

From  the  economic  perspective  of  the  employer,  the  worse  error  is  to  hire  a  poor 
worker.  Therefore,  the  sensitivity  analysis  is  sufficient  for  cost-efficient  recruiting. 
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3. 1.6  The  Spearman  Rank-Order  Correlation  Coefficient 


The  Spearman  rank-order  correlation  coefficient  is  used  to  correlate  two 
continuous  variables  that  are  measured  at  the  ordinal  level  of  measurement.  If  the 
subjects  are  rank-ordered  (from  highest  to  lowest)  on  each  of  the  two  variables  and  the 
ranks  are  correlated  using  the  Pearson  correlation  coefficient,  the  resulting  number  is  a 
Spearman  rank-order  correlation  coefficient.  The  Spearman  rank-order  correlation 
coefficient  is  used  to  evaluate  the  rank-ordered  on  the  estimated  criterion  using  different 
methods.  The  correlation  coefficient  will  be  computed  for  the  ranking  of  the  criterion 
and  the  ranking  of  the  estimated  criterion. 

3.1.7  Cross-  Val idation 

In  cross-validation,  the  data  are  split  into  two  or  more  subsets.  One  of  the  subsets 
is  called  the  construction  set,  and  is  used  for  estimation.  Predictions  for  the  cases  in  the 
other  subset,  called  the  validation  set,  can  be  obtained  from  the  model  fit  to  the 
construction  set  using  predictor  values  from  the  validation  set.  These  predictions  can  be 
compared  to  the  observed  values  of  the  response. 

However,  cross-validation  is  known  to  have  significant  restrictions.  In  particular, 
a  significant  loss  of  information  can  be  expected  when  all  available  data  are  not  used  for 
the  purpose  of  parameter  estimation.  When  the  sample  size  is  large,  this  loss  is  most 
likely  minor.  But  for  a  moderate-size  dataset,  splitting  data  can  yield  seriously  unstable 
parameter  estimates.  Previous  research  has  shown  that  sample  size  and  the  ratio  of 
predictors  selected  to  the  total  in  the  set  will  affect  validity  estimation  in  the  subset 
context.  One  useful  criterion  function  to  determine  the  outcome  of  the  cross-validation  is 
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the  square  root  of  the  average  square  error  of  prediction  (PRESS);  thus,  a  good  model 
indicates  small  value  of  PRESS  (Weisberg,  1985).  It  is  expected  that  the  value  of  PRESS 
for  the  method  of  using  both  general  and  specific  factors  is  smaller  than  that  for  the 
method  of  using  the  general  factor  only. 

For  the  simulation  study  as  shown  in  Figure  3,  the  selected  sample  (A(y)  is  split 
into  two  subsets  (e.g.,  sample  1  and  sample  2).  The  construction  set  (sample  1)  is  used 
for  identification  of  the  model  (estimation  of  regression  slopes)  and  the  validation  set 
(sample  2)  is  used  for  estimation  of  criterion.  For  the  validation  set,  PRESS  and 
Spearman  rank-order  correlation  are  calculated  according  to  the  hierarchical  factor  model 
is  +  s)  and  the  single-factor  model  (gj).  Double  cross-validation  is  carried  out  in  this 
report  so  the  procedure  outlined  above  is  applied  twice  (sample  2  is  used  for  estimation 
and  sample  1  is  used  for  prediction).  So,  for  each  sample,  the  regression  equation  and  the 
predicted  criterion  are  calculated.  If  the  results  of  double  cross-validation  are  close,  as 
suggested  by  Pedhazur  (1982),  we  may  combine  the  samples  and  calculate  the  regression 
equation  to  be  used  in  prediction.  The  purpose  of  this  method  is  to  study  the  differences 
of  Spearman  rank-order  correlation  coefficients  as  well  as  the  differences  of  PRESS 
across  two  models  in  the  context  of  double  cross-validation. 

3.2  The  Monte  Carlo  Study 

This  report  aims  to  examine  the  quality  of  factor  scores  as  predictors  to  improve 
the  precision  of  selection  and  classification.  The  classic  model  says  that  the  general 
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factor  will  account  for  almost  all  the  relevant  true  score  covariances  among  observed 
measures.  It  uses  only  general  ability  as  a  predictor  of  future  performance  or  as  a 
selection  criterion  to  screen  applicants.  On  the  other  hand,  the  multiple-factor  model 
assumes  that  performance  is  multidimensional  and  is  composed  of  a  number  of  basic 
components,  which  are  such  that  some  people  may  perform  well  on  one  component  but 
not  so  well  on  others.  In  this  report  the  specific  factors,  in  addition  to  the  general  factor, 
are  included  as  predictors.  The  regression  method  is  used  to  estimate  faetor  scores 
because  it  is  a  linear  transformation  of  observed  variables.  The  selection  is  based  on 
factor  scores  so  as  to  give  unbiased  estimates  of  regression  slopes.  The  research  question 
is:  Does  the  method  of  prediction  of  using  general  and  specific  factor  scores  perform 
better  than  the  method  of  prediction  of  using  the  general  factor  score  only? 

A  simulation  model  is  developed  to  explore  the  use  of  factor  scores  in  prediction 
under  conditions  varying  the  following:  (1)  the  factor  loadings  for  the  specific  factor  in 
terms  of  high  vs.  low  determinacy;  (2)  the  regression  slopes  for  the  eriterion  regressed  on 
general  and  specific  factors;  (3)  the  selection  methods;  (4)  the  R  square;  (5)  the  selection 
ratio;  and  (6)  the  sample  size  for  the  selected  subjects.  Next,  the  prediction  of  future 
performance  is  examined  in  terms  of  sensitivity,  specificity,  and  proportion  of  correct 
decisions.  In  addition,  two  approaches,  cross-validation  and  Spearman  rank-order 
correlation,  are  used  to  evaluate  the  accuracy  of  prediction. 

The  Monte  Carlo  study  is  intended  to  answer  the  following  questions:  How  does 
the  quality  of  factor  scores  affect  the  selection  and  prediction?  Does  the  inclusion  of  the 
specific  factor  enhance  the  predictive  validity?  Does  the  selection  based  on  the  specific 
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factor  in  addition  to  the  general  factor  increase  sensitivity,  specificity,  proportion  of 
correct  decisions,  and/ or  rank  correlation?  How  reliable  must  factor  scores  be  in  order  to 
improve  predictions?  How  do  the  R  square,  the  regression  slope,  the  selection  ratio,  and 
the  sample  size  affect  the  predictive  validity? 

3. 2. 1  Design  of  the  Monte  Carlo  Study 

The  design  of  this  Monte  Carlo  study  is  summarized  in  Table  1.  iThis  report 
considers  4  datasets  (2  selection  ratios  x  2  sample  sizes  for  the  selected  individuals)  and 
36  latent  variable  models  (12  models  x  3  cases),  resulting  in  144  combinations.  The  three 
cases  are  named  as  g  +  1  5,  g-  +  2  and  g  +  3  s,  where  “g”  represents  the  general  factor 
and  “5”  represents  the  specific  factor.  The  notation  of  g  +  3  5  means  one  general  factor 
plus  three  specific  factors  (sj,  S2,  and  55).  For  each  of  the  144  conditions,  one  hundred 
replications  are  performed. 

Table  2  shows  the  parameter  values  for  each  model  across  the  three  different 
cases.  The  standardized  factor  loadings,  variance  of  factors  and  criterion,  regression 
slopes,  standardized  regression  slopes,  and  R  square  are  presented. 

Selected  Sample  Size  and  Selection  Ratio 

In  order  to  obtain  the  actual  prediction  situations,  two  different  sample  sizes  for 
the  selected  individuals  are  studied.  Selection  ratios  of  0.50  and  0.10  are  used  in  the 
simulation.  The  sample  sizes  for  the  selected  subjects  are  250  and  500,  corresponding  to 
the  applicant  sample  sizes  of  500  and  1,000  with  a  50%  selection  ratio,  and  the  applicant 
sample  sizes  of  2,500  and  5,000  with  a  10%  selection  ratio. 
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Selection  Methods 


In  the  simulated  data,  the  criterion  variable  is  available  for  all  applicants  so  the 
true  selectee  is  determined  by  the  observed  criterion  (y)  and  the  selection  ratio.  Two 
selection  methods  are  used:  the  estimated  general  factor  score  {gj),  and  the  predicted 
criterion  (y)  that  is  estimated  by  general  and  specific  factor  scores  (g  and  5).  These 
factor  scores  are  computed  using  the  regression  method  for  all  applicants.  Given  that 
previous  research  (e.g.,  Ree  &  Earles,  1991;  Earles  &  Ree,  1992;  Ree,  Earles,  & 
Teachout,  1994;  Ree  &  Carretta,  1995)  uses  the  general  ability  as  selection  criterion,  the 
current  study  will  use  the  same  selection  criterion  as  the  standard  for  comparison.  The 
classification  of  applicants  as  accepted  or  rejected  is  based  only  on  the  ranking  of  the 
estimated  general  factor  score.  In  contrast,  the  decision  method  used  in  this  report  is 
based  on  the  predicted  criterion  (y)  by  using  both  general  and  specific  factor  scores  as 
predictors.  The  purpose  is  to  compare  the  efficiency  of  these  two  selection  methods. 

Two  types  of  latent  variable  models  as  shown  in  Figures  4  and  5  will  be 
considered  as  examples  for  the  g  +  1  s'  case.  The  first  type  is  a  single-factor  model  that 
has  only  one  general  factor  (g/).  The  second  type  is  a  hierarchical  factor  model  that  has  a 
general  factor  (g)  which  influences  all  of  the  subtests  (xj  to  x]q)  as  well  as  one  specific 
factor  (s)  which  influences  some  of  the  subtests  (x^  to  xjq).  In  Figures  4  and  5,  x;  to  X70 
are  the  observed  variables  and  y  is  the  observed  criterion  variable.  For  the  single-factor 
model,  the  criterion  variable  is  influenced  by  one  general  factor  (g;),  which  represents 
general  ability.  For  the  hierarchical  factor  model,  the  criterion  variable  is  influenced  by 
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one  general  factor  (g),  which  represents  general  ability,  and  one  specific  factor  (s),  which 
represents  a  special,  narrow  ability.  The  factors  are  all  uncorrelated  with  each  other. 
Since  the  hierarchical  factor  models  for  g  +  2  5  and  g  +  3  s'  are  similar  to  that  for  g  +  1  s, 
they  are  not  presented  here. 

True  Models  ; 

Table  1  shows  the  36  model  combinations.  Models  1  through  6  are  used  to 
differentiate  the  effects  of  predictive  validity  for  g  and  s  and  also  specific  factor 

.  u 

determinacy,  while  controlling  for  the  value  of  (0.4)  and  general  factor  loadings. 
Models  7  through  12  have  the  same  pattern  as  described  in  models  1  through  6,  but  the 
value  of  is  set  at  0.6. 

The  range  of  g  factor  loadings  is  from  0.25  to  0.80  for  all  12  models  in  each  of  the 
three  cases.  The  range  of  s  factor  loadings  for  the  case  of  g  +  1  \  is  from  0.18  to  0.54  in 
models  1,  3,  5,  7,  9,  and  11,  and  from  0.36  to  0.72  for  models  2,  4,  6,  8, 10,  and  12.  In 
models  1  and  2,  the  standardized  regression  slope  for  g  (0.54)  is  much  higher  than  that  for 
s  (0.32)  and  the  variances  of  g  and  5  are  set  at  1.00  and  0.80,  respectively,  while  in 
models  3  and  4,  standardized  regression  slopes  for  g  (0.45)  and  s  (0.45)  are  nearly 
equivalent  and  the  variances  of  g  and  5  are  set  at  1.00  and  0.81,  respectively.  In  models  5 
and  6,  the  standardized  regression  slope  for  g  (0.32)  is  less  than  that  for  5  (0.55)  and  the 
variances  of  g  and  5  are  set  at  1.00  and  0.81,  respectively.  Models  7  through  12  have  the 
same  pattern  of  regression  slopes  as  described  for  models  1  through  6. 

For  the  g  +  2  .s  case,  the  range  of  low  specific  factor  loadings  is  from  0.13  (0.12) 
to  0.54;  and  the  range  of  high  specific  factor  loadings  is  from  0.32  (0.30)  to  0.72.  In 
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models  1  and  2,  the  standardized  regression  slope  for  g  (0.49)  is  much  higher  than  that  for 
sj  (0.33)  and  S2  (0.23)  and  the  variances  of  g,  sj,  and  S2  are  set  at  1.00,  0.80,  and  0.40, 
respectively,  while  in  models  3  and  4,  standardized  regression  slopes  for  g  (0.36),  5 ; 
(0.36),  and  s2  (0.37)  are  nearly  equivalent  and  the  variances  of  g,  sj,  and  $2  are  set  at 
1.00,  0.81,  and  0.36,  respectively.  Furthermore,  in  models  5  and  6,  the  standardized 
regression  slope  for  g  (0.32)  is  less  than  that  for  s]  (0.39)  and  S2  (0.39)  and  the  variances 
of  g,  s],  and  S2  are  set  to  1.00,  0.81,  and  0.36,  respectively.  The  parameters  for  models  7 
through  12  follow  the  same  pattern  described  in  models  1  through  6. 

For  the  g  +  3  case,  the  two  sets  of  range  for  specific  factor  loadings  are  from 
0.23  (0.24)  to  0.45  and  from  0.23  (0.24)  to  0.36.  In  models  1  and  2,  the  standardized 
regression  slope  for  g  (0.43)  is  much  higher  than  that  for  sj  (0.38)  ,  sj  (0.20),  and 
(0.16)  and  the  variances  of  g,  sj,  S2,  and  53  are  set  at  1.00,  0.80,  0.60,  and  0.40, 
respectively;  in  models  3  and  4,  standardized  regression  slopes  for  g  (0.32),  sj  (0.32),  S2 
(0.31),  and  53  (0.32)  are  nearly  equivalent  and  the  variances  of  g,  sj,  S2,  and  S3  are  set  at 
1.00,  0.81,  0.64,  and  0.36,  respectively;  in  models  5  and  6  standardized  regression  slope 
for  g  (0.31)  is  less  than  that  for  sj  (0.37),  S2  (0.49),  and  S3  (0.37)  and  the  variances  of  g, 
s],  S2,  and  S3  are  set  at  1.00,  0.81,  0.64,  and  0.36,  respectively  .  Models  5  through  12 
have  a  combination  of  regression  slopes  and  factor  variances  similar  to  that  seen  in 
models  1  through  6. 

Factor  Determinacv 

It  is  clear  that  increasing  the  number  of  observed  variables  increases  the  reliability 
of  the  factor  score  measurement  (Acito  &  Anderson,  1986;  Short,  1990).  In  this 
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simulation,  the  effects  on  specific  factor  determinacies  are  investigated  by  varying  the 
factor  loadings.  When  the  general  factor  loadings  are  held  constant,  increasing  the 
specific  factor  loadings  should  increase  the  reliability  of  the  specific  factor.  Two  sets  of 
standardized  factor  loadings  for  specific  factors  (low  X  5  vs.  high  X  j)  are  used  to 
investigate  the  issue  of  determinacy. 

As  expected,  increasing  the  size  of  loadings  increases  the  factor  determinacy.  The 
determinacy  gives  information  about  the  reliability  of  the  factor.  In  the  current  study,  the 
general  factor  determinacy  is  about  0.94  for  the  case  of  g  +  1  fi-om  0.90  to  0.92  for  the 
case  of  g  +  2  5,  and  from  0.90  to  0.91  for  the  case  of  g  +  3  5-  when  general  factor  loadings 
are  firom  0.25  to  0.80. 

In  the  g  +  1  5  case,  for  models  2,  4,  6,  8,  10,  and  12  with  high  specific  factor 
loadings  (ranging  from  0.50  to  0.80),  the  specific  factor  determinacy  is  about  0.85.  In 
contrast,  for  the  odd-numbered  models  with  low  specific  factor  loadings  (ranging  from 
0.20  to  0.60),  the  specific  factor  determinacy  is  about  0.70. 

In  the  case  of  g  +  2  5,  models  with  high  specific  factor  loadings  (ranging  from 
0.50  to  0.80)  have  the  specific  factor  determinacies  of  about  0.84  and  0.71  for  models  2 
and  8;  and  0.85  and  0.68  for  models  4,  6,  10,  and  12.  In  contrast,  models  with  low 
specific  factor  loadings  (ranging  from  0.20  to  0.60)  have  the  specific  factor  determinacies 
of  about  0.68  and  0.53  for  models  I  and  7;  and  0.68  and  0.51  for  models  3,  5,  9,  and  11. 

In  the  case  of  g  +  3  5,  for  models  2,  4,  6,  8  10,  and  12  with  high  specific  factor 
loadings  (ranging  from  0.30  to  0.80),  the  specific  factor  determinacies  are  about  0.79, 
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0.76,  and  0.75.  In  contrast,  for  models  1,  3,  5,  7,  9,  and  11  with  low  specific  factor 
loadings  (ranging  from  0.30  to  0.70),  the  specific  factor  determinacies  are  about  0.62, 
0.72,  and  0.61. 

3.2.2  Analyses 

The  following  steps  are  carried  out  in  the  Monte  Carlo  study: 

(1)  generation  of  observed  variables  (xj  to  Xjo),  criterion  variable  (y),  and  true 
factor  scores  on  general  (g)  and  specific  (s)  factors  for  N  applicants  as  shown  in  Figure  6; 

(2)  estimation  of  factor  score  determinacies; 

(3)  estimation  of  parameters  ("F  and  A)  based  on  the  single-factor  model  and  the 
hierarchical  factor  model; 

(4)  estimation  of  factor  scores  (g/,  g ,  and  s )  for  each  individual; 

(5)  obtaining  a  selected  sample  (A]y)  according  to  the  estimated  general  factor 
scores  (g/)  and  the  selection  ratio; 

(6)  estimation  of  regression  slopes  (pg  and  p^)  for  general  and  specific  factors 
with  the  criterion  using  the  selectees  T/y  from  the  previous  step; 

(7)  estimation  of  the  predicted  criterion  ( j) )  using  the  estimated  regression  slopes 
and  the  estimated  factor  scores  (this  new  selection  method  is  defined  as  the  equation 
below: 

i)  =  a  +  pg  xg+p^x.y); 

(8)  obtaining  a  new  selected  sample  based  on  y  and  the  selection  ratio; 
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(9)  categorizing  applicants  as  success  or  failure  based  on  the  true  criterion  y  and 
selection  ratio  ; 

(10)  computing  sensitivity  and  specificity  for  the  classifications  of  the  true 
criterion  (y)  and  the  predicted  criteria  (y^^  vs.  y^+^ ); 

(11)  computing  the  Spearman  rank-order  correlation  coefficient  between  the  rank- 
ordered  y  and  the  rank-ordered  ; 

(12)  computing  the  Spearman  rank-order  correlation  coefficient  l^etween  the  rank- 
ordered  y  and  the  rank-ordered  yg+f; 

(13)  double  cross-validation: 

a)  splitting  of  the  selected  sample  into  two  subsets  (labeled  as  sample  1  and 

sample  2),  the  construction  set  for  estimation  and  the  validation  set  for  prediction  (see 
Figure  3),  ; 

b)  estimation  of  the  criterion  function,  the  square  root  of  the  average  squared  error 
of  prediction,  the  equation  of  PRESS  is  defined  in  Figure  3, 

c)  computing  of  the  Spearman  rank-order  correlation  coefficient  between  the 
rank-ordered  and  the  rank-ordered  y, 

d)  repeating  steps  a  through  c,  where  sample  2  is  for  estimation  and  sample  1  is 
for  prediction. 
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CHAPTER 4 


RESULTS  OF  THE  MONTE  CARLO  STUDY 


In  this  chapter,  the  results  of  the  Monte  Carlo  study  are  presented.  The  purpose  of 
the  Monte  Carlo  study  is  to  examine  the  quality  of  factor  scores  as  predictors  to  improve 
the  precision  of  selection  and  classification  of  applicants.  First,  the  results  of 
classification  and  Spearman  rank-order  correlation  coefficient  are  described  and 
summarized.  Next,  the  results  are  described  as  a  function  of  selection  methods,  selection 
ratio,  and  sample  size.  The  effects  of  model  features  in  terms  of  R  square,  standardized 
regression  slope,  and  factor  loadings  are  then  presented.  .The  last  section  shows  the 
increment  of  prediction  from  the  specific  factors. 

4.1  Results  of  Sensitivity.  Specificity,  and  Proportion  of  Correct  Decisions 

The  analysis  results  are  summarized  in  Tables  3  through  5  for  g  +  1  s,  g  +  2  and 
g  +  3  5,  where  the  classifications  in  different  combinations  of  sample  sizes  for  the 
selected  individuals,  selection  ratios,  models,  and  selection  methods  are  shown.  The 
pattern  of  results  is  virtually  identical  for  all  of  the  models,  so  only  one  set  of  results  is 
shownforg+  1  5, g  + 2 5 ,  andg  + 3  5. 
s  +  1  s 

Model  1  in  the  first  section,  defined  as  N-250,  R=0.5  in  Table  3,  shows  the  50% 
selection  ratio  (R=0.5)  in  the  sample  of  500  applicants  (250/0.5=500).  The  first  block  of 
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gj  presents  selection  results  using  the  method  in  which  the  predictive  criterion  is  based 
on  the  estimated  general  factor  score  {gj).  The  results  show  that  68.75%  of  the 
successful  subjects  were  also  selected  by  the  predictive  criterion  using  gj,  but  that 
31.25%  (1  -  68.75%)  of  the  successful  subjects  were  excluded  by  this  criterion.  The 
results  show  that  about  156  subjects  (500  x  31.25%),  or  31.25%  of  the  applicants,  are 
misclassified  by  the  traditional  method  {gj).  In  contrast,  the  second  block  of  |  +  1  5  in 
Table  3,  using  both  estimated  general  and  specific  factors  (|  +  1  5),  shows  that  69.62% 
of  the  successful  subjects  are  selected  according  to  the  predictive  criterion  (j)).  Of  the 
successful  subjects,  30.38%  are  not  selected  by  the  new  criterion.  The  results  show  that 
about  150  subjects,  or  30.38%  of  the  applicants,  are  misclassified  by  the  new  method. 
The  third  block  of  difference  shows  the  increase  in  sensitivity,  specificity,  and  proportion 
of  correct  decisions  that  occurs  when  using  specific  in  addition  to  general  factors  as 
predictors.  It  shows  that  there  is  an  increase  of  0.87%  for  sensitivity,  specificity,  and 
proportion  of  correct  decisions  in  model  1  with  a  50%  selection  ratio  and  sample  size  250 
for  the  selected  subjects. 

Model  8  in  the  second  section,  defined  as  N=250,  R=0.1  in  Table  4,  displays  the 
10%  selection  ratio  (R=0.1)  in  the  sample  of  2,500  applicants  (250/0.1).  The  results 
show  that  47.88%  of  the  successful  subjects  were  also  selected  by  the  predictive  criterion 
using  gj,  but  52.12%  of  the  successful  subjects  were  excluded  by  this  criterion.  There 
are  about  260  subjects,  or  10.42%  (1  -  98.589%)  of  the  applicants,  who  are  misclassified 
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by  the  gj  method.  In  contrast,  using  both  estimated  general  and  specific  factors  (g ,  sj, 
S2 ,  and  )  shows  that  50.39%  of  the  successful  subjects  are  chosen  according  to  the 
predictive  criterion  ( j)).  Of  the  successful  subjects,  49.61%  are  not  selected  by  the  new 
criterion.  There  are  about  248  subjects,  or  9.92%  of  the  applicants,  who  are  misclassified 
by  the  g  +  3  5  method.  Furthermore,  there  is  ah  increase  of  2.50%  for  sensitivity,  an 
increase  of  0.28%  for  specificity,  and  an  increase  of  0.50%  for  the  proportion  of  correct 
decisions. 

Model  12  in  the  fourth  section,  defined  as  N=500,  R=0.1  in  Table  5,  shows  the 
10%  selection  ratio  (R=0.1)  in  the  sample  of  5,000  applicants  (500/0.1).  The  first  block 
8l  presents  selection  results  using  the  method  in  which  the  predictive  criterion  is 
based  on  the  estimated  general  factor  score  (g/).  The  results  show  that  34.97%  of  the 
successful  subjects  were  also  selected  by  the  predictive  criterion  using  gj ,  but  65.03%  of 
the  successful  subjects  were  excluded  by  this  criterion.  Because  the  criterion  information 
is  observed  for  all  applicants,  the  results  show  that  about  650  subjects  or  13.01%  (1  - 
86.99%)  of  the  applicants  are  misclassified  by  g/  method.  In  contrast,  the  second  block 
of  g  +  3  5  in  Table  5,  using  both  estimated  general  and  specific  factors  (g,  Sj,  $2,  and 
),  shows  that  44.39%  of  the  successful  subjects  are  selected  according  to  the  predictive 
criterion  (j)).  Of  the  successful  subjects,  55.61%  are  not  selected  by  the  new  criterion. 
The  results  show  that  about  556  subjects,  or  1 1.12%  of  the  applicants,  are  misclassified 
by  the  g  +  3  i  method.  The  third  block  shows  that  there  is  an  increase  of  9.42%  for 
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sensitivity,  an  increase  of  1.05%  for  specificity,  and  an  increase  of  1.88%  for  the 
proportion  of  correct  decisions  in  model  1 2  with  a  1 0%  selection  ratio  and  sample  size 
500  for  the  selected  subjects. 

Figures  7  through  9  present  the  plots  of  sensitivity,  specificity,  and  proportion  of 
correct  decisions  across  four  combinations  of  sample  sizes  for  the  selectees  and  selection 
ratios  for  g  +  \  s,  g  +  2  and  g  +  ^3  s,  respectively.  Each  plot  shows  two  selection 
methods  and  displays  by  ordering  the  new  method  (general  plus  specific  factors).  Figures 

V  ■  t-  f 

10  through  12  show  the  difference  in  sensitivity,  specificity,  and  proportion  of  correct 
decisions  between  the  method  of  using  the  general  factor  only  as  predictor  and  the 
method  of  using  the  general  and  specific  factors  as  predictors  for  g  +  1  5,  g  +  2  5,  and  g  + 
3  s  cases,  respectively. 

4.2  Results  of  Spearman  Rank-Order  Correlations 

Spearman  rank-order  correlations  are  obtained  from  the  total  applicant  sample  and 
the  cross-validation  sample. 

4.2.1  Applicant  Samnle 

The  Spearman  rank-order  correlation  coefficients  between  the  rank-ordered  true 
criterion  and  the  rank-ordered  estimated  criterion  are  given  in  Tables  3  through  5  for  the 
cases  ofg  +  1  .y,g  +  2  5,  and  g  +  3  5,  respectively.  These  coefficients  are  computed  for 
all  applicants.  Figures  13  through  15  present  the  plots  in  ascendant  order  for  the  new 
approach.  The  results  agree  with  the  analyses  of  sensitivity,  specificity,  and  proportion  of 
correct  decisions.  It  is  observed  that  the  pattern  of  results  for  rank  correlation  is  similar 
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to  that  for  classification.  Moreover,  the  results  are  not  affected  by  selection  ratio  and 
sample  sizes  for  the  selected  subjects.  However,  the  method  of  using  the  specific  factor 
in  addition  to  the  general  factor  to  predict  the  rank-ordered  criterion  performs  better  than 
the  method  of  using  the  general  factor  alone  to  predict  the  rank-ordered  criterion. 

4.2.2  Cross-Validation  Sample 

The  summary  of  cross-validation  is  shown  in  Tables  6  through  8.^  Figures  16 
through  18  also  display  the  difference  of  Spearman  rank-order  correlation  and  the 

V  •  *•  '* 

difference  of  PRESS  for  each  of  the  subsamples  by  using  different  selection  methods. 
The  comparison  between  these  two  subsamples  (labeled  as  sample  1  and  sample  2)  is 
presented  in  these  figures. 

As  expected,  the  value  of  the  rank-order  coefficient  obtained  in  the  subsample  of 
cross-validation  is  less  than  that  attained  in  the  total  applicant  sample.  It  is  important  to 
note  that  the  value  of  difference  in  the  cross-validation  is  much  greater  than  the  value  of 
difference  obtained  in  the  applicant  sample. 

Furthermore,  examining  the  values  of  PRESS  across  sample  1  and  sample  2,  it 
shows  that  the  values  in  the  hierarchical  models  are  smaller  than  those  in  the  single-factor 
models.  These  values  of  PRESS  indicate  that  the  prediction  of  using  general  and  specific 
factors  as  predictors  results  in  smaller  amount  of  prediction  error.  In  summary, 
comparisons  across  the  subsamples,  the  results  of  rank  correlation  and  PRESS  are  stable. 
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4. 3  Effects  of  Selection  Methods 

In  summary.  Tables  3  through  5  and  Figures  7  through  1 5  show  that  the  prediction 
approach  of  using  general  and  specific  factor  scores  performs  better  than  the  method  of 
using  an  estimated  general  factor  score  alone.  By  introducing  the  specific  factor  as 
predictor,  this  approach  not  only  increases  the  precision  of  prediction  in  relation  to  the 
Spearman  rank-order  correlation,  but  also  tends  to  produce  efficient  classification  in 
terms  of  sensitivity,  specificity,  and  proportion  of  correct  decisions. 

V 

The  results  for  cross-validation  are  similar  to  the  results  described  above  except 
for  a  few  cases  which  show  a  negative  difference.  It  is  observed  that  the  rank  coefficients 
obtained  from  cross-validation  are  smaller  than  those  obtained  from  the  applicant  sample 
but  the  increment  is  more  notable  for  the  results  of  cross-validation. 

4. 4  Effects  of  Selection  Ratio  and  Sample  Size 

Given  a  particular  sample  size  for  the  selected  subjects,  when  the  selection  ratio 
increases,  the  proportion  of  successful  individuals  who  are  accepted  (sensitivity) 
increases,  the  proportion  of  failing  individuals  who  are  rejected  (specificity)  decreases, 
and  the  proportion  of  correct  decisions  decreases.  On  the  basis  of  this  simulation,  when 
the  selection  ratio  decreases  the  difference  in  sensitivity  between  the  two  methods 
increases. 

Comparing  two  sets  of  plots  with  the  same  number  of  selectees  (N=250  or 
N=500)  in  Figure  10,  the  results  show  that  the  lower  selection  ratio  leads  to  the  larger 
increase  of  difference  in  sensitivity  and  to  the  smaller  increase  of  difference  in  specificity 
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and  proportion  of  correct  decisions.  The  same  phenomena  as  described  above  are 
observed  in  Figures  1 1  and  12.  Figures  7  through  9  also  present  information  as  to  how 
the  selection  ratio  affects  the  results  of  classification. 

Two  different  sample  sizes  for  the  selected  subjects  were  studied  to  investigate  the 
effect  of  sample  size  on  prediction.  The  results  show  that  the  sample  size  for  the 
selectees  does  not, affect  sensitivity,  specificity,  and  proportion  of  correct  decisions.  In 
comparing  two  sets  of  plots  with  the  same  selection  ratio  in  Figures  7  through  12,  we 

5.- 

observe  that  increasing  the  number  of  selectees  does  not  change  the  results  of 
classifications.  The  patterns  of  these  results  are  very  similar. 

In  the  cross-validation,  the  sample  size  is  decreased  to  125  and  250  for  each 
subsample  to  compute  rank  coefficient.  As  expected,  the  coefficients  are  smaller  than 
those  obtained  from  analyzing  all  applicants;  in  contrast,  the  difference  in  relation  to  the 
increment  from  the  specific  factor  becomes  more  notable.  It  is  concluded  that  selection 
ratio  and  sample  size  for  the  selected  subjects  have  effects  on  rank  coefficients.  As 
sample  size  for  the  selected  subjected  or  selection  ratio  increases,  Spearman  rank 
correlation  coefficient  increases.  While  the  increment  of  difference  depends  on  the 
increase  in  sample  size  for  the  selected  subjects  and  the  decrease  of  selection  ratio. 
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4.5  Effects  of  Models 

Three  elements,  R  square,  standardized  regression  slope,  and  determinacy, 
generate  12  (2  x  3  x  2)  different  conditions  for  each  of  the  three  cases.  The  following 
section  addresses  the  effect  for  each  factor  separately. 

4.5.1  R  Square  ,  ^ 

It  is  important  to  note  that  high  value  will  enhance  the  precision  of  selection 
and  classification.  The  results  show  that  the  classifications  from  models  with  an  R^  value 
of  0.6  are  more  accurate  than  those  with  an  R^  value  of  0.4.  The  high  value  of  R^ 
increases  the  value  of  the  Spearman  rank-order  correlation  coefficient  in  both  the 
applicant  sample  and  the  cross-validation  sample. 

4.5.2  Standardized  Regression  Slones 

As  expected,  controlling  for  the  factor  loadings  and  R^,  the  accuracy  of 
classification  by  using  multiple  predictors  is  influenced  by  the  value  of  the  regression 
slope  on  the  specific  factor  relative  to  that  of  the  general  factor.  Considering  models  1,3, 
and  5  (see  Figure  7),  the  values  of  sensitivity,  specificity,  and  proportion  of  correct 

decisions  are  in  descendant  order  as  Pg*  >p^*  (model  1),  pg*  »  p^*  (model  3),  and 
♦  ♦ 

Pg  <  Pj  (model  5).  Using  models  8,  10,  and  12  as  another  example,  the  same  pattern 

is  observed  in  that  model  8  has  the  highest  values  of  sensitivity,  specificity,  and 
proportion  of  correct  decisions. 
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When  comparing  the  classification  using  only  the  general  factor  score  to  the 
classification  using  both  general  and  specific  factor  scores,  it  may  be  noted  that  the 
greatest  gain  from  the  method  of  using  general  and  specific  factor  scores  is  found  in 

model  12,  which  contains  a  high  value  of  .  If  the  values  of  and  factor  loadings  are 

held  constant,  the  effects  of  P*  could  be  examined  among  models  1,3,  and  5;  or  models 
2, 4,  and  6;  or  models  7,  9,  and  1 1;  or  models  8,  10,  and  12.  It  is  observed  that  in  the  case 

of  p*g  <  p  \ ,  the  prediction  of  using  specific  factors  in  addition  to  the  general  factor  does 
much  better  than  the  prediction  without  using  specific  factors  as  predictors.  The  results 
of  classification  are  similar  to  the  results  of  Spearman  rank-order  correlation  among  these 
model  comparisons. 

The  same  phenomenon  is  observed  in  the  results  of  cross-validation.  Specifically, 
for  models  with  p  g  <  p  j ,  the  differences  of  Spearman  rank-order  correlations  between 
^  +  5  and  gj  are  the  greatest.  Comparisons  of  ^  -t-  5  to  gy  in  the  case  of  g  -i-  3  5  and 

N=500,  R=0.1  in  Table  8,  model  12  (P*g  <  p*j)  results  in  cin  increase  of  0.274  and  0.282 
for  Spearman  rank-order  correlation  in  sample  1  and  sample  2,  respectively;  the 

increments  for  model  8  (p*g  >p%)  are  0.068  and  0.059  in  sample  1  and  sample  2, 

respectively;  the  increments  for  model  10  (P*g  «  p%)  are  0.220  and  0.223  in  sample  1 
and  sample  2,  respectively. 


44 


4. 5. 3  Factor  Loadin  2s 

When  the  regression  slopes  and  the  value  of  are  held  constant,  the  value  of 
factor  loadings  (factor  determinacy)  will  affect  the  prediction  in  terms  of  sensitivity, 
specificity,  proportion  of  correct  decisions,  and  Spearman  rank-order  correlation 
coefficient.  A  higher  value  of  factor  determinacy  will  improve  the  accuracy  of 
prediction.  The  same  gffects  ofTactor  loadings  can  be  observed  in  the  results  of  cross- 
validation. 

.  ;  r 

4 . 6  The  Increment  of  Prediction  From  the  Specific  Factors 

The  increment  of  prediction  from  the  specific  factors  is  evaluated  by  the 
classification  and  Spearman  rank-order  correlation  coefficient  for  the  applicants.  The 
approach  of  cross-validation  is  also  used  for  detecting  the  same  effects  on  Spearman 
rank-order  correlation  coefficient, 

4.6.1  The  Gains  in  Classification 

Looking  at  the  results  for  g  +  1  5,  it  is  noted  that  the  range  of  difference  for 
sensitivity  is  from  0.39%  to  1 1.58%;  the  range  of  difference  for  specificity  is  from  0.09% 
to  6.48%;  and  the  range  of  difference  for  proportion  of  correct  decisions  is  from  0.16%  to 
6.48%.  In  the  situations  with  a  10%  selection  ratio,  the  largest  increment  of  sensitivity  is 
about  1 1%  in  models  1 1  and  12,  and  the  second  largest  increase  is  about  7%  in  models  5 
and  6.  There  is  a  similarity  among  these  four  models.  That  is,  the  regression  slope  for 
the  specific  factor  is  greater  than  the  weight  for  the  general  factor.  It  is  observed  that 
models  1, 2,  7,  and  8  have  a  smaller  amount  of  increase  for  sensitivity.  When  the  specific 


45 


factor  with  low  regression  slope,  the  difference  of  sensitivity  between  the  gj  method  and 
the  g  +  1  5  method  decreases.  For  models  with  equivalent  slopes  for  general  and 
specific  factors  (models  3,  4,  9,  and  10),  the  gain  of  sensitivity  is  about  3%  to  4%  for  an 
value  of  0.4  and  5%  to  6%  for  an  value  of  0.6. 

In  Figure  7,  each  plot  is  presented  by  ordering  the  value  of  sensitivity,  specificity, 
and  proportion  of  correct  decisions  for  g  +  1  5  among  the  twelve  models.  As  expected, 
model  5  has  the  lowest  value  of  sensitivity,  specificity,  and  proportion  of  correct 

.  f  r 

decisions.  In  contrast,  model  8  has  the  highest  value  of  sensitivity,  specificity,  and 
proportion  of  correct  decisions.  The  prediction  model  with  low  R^,  low  determinacy  on 
specific  factor,  and  low  regression  weight  on  specific  factor  tends  to  produce  the  least 
accurate  results  of  selection  and  classification.  If  the  specific  factor  plays  an  important 
role  in  the  future  prediction,  the  strategy  of  including  the  specific  factor  as  predictor  will 
improve  the  accuracy  of  selection  and  classification. 

Similarly,  the  pattern  of  results  for  g  +  2  5*  and  g +  3  s  is  obtained  as  that  for  g  +  1 
s.  Here,  model  5  for  g  +  1  .s*  (see  Table  3)  is  chosen  to  be  an  example  for  demonstration. 
The  difference  shows  an  increase  of  4%  of  the  successful  applicants  who  should  be 
selected  in  the  cases  of  50%  selection  ratio,  while  in  the  cases  using  a  10%  selection  ratio, 
the  difference  increases  7%  of  the  successful  applicants  who  should  be  selected. 

4.6.2  The  Gains  in  Spearman  Rank-Order  Correlation  CoefTicient 

The  range  of  difference  for  the  Spearman  rank-order  correlation  is  from  0.010  to 
0.164  for  the  g  +  1  5  case,  from  0.005  to  0.072  for  the  g  +  2  .y  case,  and  from  0.002  to 
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0.132  for  the  g  +  3  s  case  in  the  applicant  sample.  In  addition,  for  the  subsamples  in 
cross-validation,  the  range  of  difference  for  the  Spearman  rank-order  correlation  is  from 
0.013  to  0.314  for  the  g  I  s  case,  from  ’-0.003  to  0.185  for  the  g"  +  2  5  case,  and  from  — 
0.008  to  0.282  for  the  ^  +  3  5  case. 
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CHAPTER  5 


REAL  DATA  APPLICATION 

The  findings  reported  from  the  simulation  study  point  to  benefits  potentially 
obtainable  from  an  effective  selection  and  classification  approach.  TTie  aim  of  this  real 
d^ata  analysis  is  to  provide  an  application  of  the  use  of  factor  scores  as  predictors  in  the 
context  of  selection  and  predictive  validity.  Analyses  are  performed  for  each  of  the  nine 
Army  Project  A  jobs  (Campbell  &  Zook,  1991,  1992)  and  two  jobs  from  the  Marine 
Corps  Job  Performance  Measurement  (JPM^)  Project  using  hands-on  job  performance  as 
the  criterion.  The  results  of  classification  are  evaluated  by  Spearman  rank-order 
correlation  coefficient,  sensitivity,  specificity,  and  proportion  of  correct  decisions. 

5.  J  The  Army  Prdject  A  and  the  Marine  Corps  JPM Data 

The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  the  test  battery 
which  the  United  States  Military  Services  have  used  since  1976  to  determine  the 
cognitive  qualification  of  applicants  for  service.  The  battery  serves  both  to  determine 
whether  applicants  meet  minimum  enlistment  standards  and  to  aid  in  determining  the 
specialty  area  in  which  an  applicant  might  most  benefit  from  advanced  training.  In  the 
Army  Project  A  and  the  Marine  Corps  JPM,  additional  tests  have  been  used  to  extend  the 
range  of  abilities  covered  by  the  ASVAB. 


*The  Marine  Corps  JPM  data  have  been  kindly  made  available 


by  Neil  B.  Carey  at  the  U.  S.  Marine  Corps. 
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5.1.1  Sample 


The  data  analyzed  for  this  report  are  from  the  nine  different  jobs  of  Batch  A  of  the 
Army  Project  A  Concurrent  Validity  Study  and  two  jobs  of  the  Marine  Corps  JPM  data. 
Table  9  lists  the  nine  Army  jobs  (Infantryman,  Cannon  Crewman,  Tank  Crewman,  Radio 
Operator,  Vehicle  Mechanic,  Motor  Transport,  Administrative,  Medical,  and  Military 
Police)  and  two  Marine  jobs  (Helicopter  Mechanic  and  Automotive  Mechanic)  and  gives 
the  number  of  soldiers  included  in  the  present  analyses. 

5.1.2  Variables 

A  listing  of  predictor  measures  is  given  in  Table  10.  It  lists  10  ASVAB  subtests, 
12  Army  Project  A  subtests,  and  8  subtests  from  the  Enhanced  Computer  Administration 
Test  (ECAT). 

The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  includes  ten  tests: 
General  Science,  Arithmetic  Reasoning,  Word  Knowledge,  Paragraph  Comprehension, 
Numerical  Comprehension,  Coding  Speed,  Auto  and  Shop  Information,  Math 
Knowledge,  Mechanical  Comprehension,  and  Electronics  Information.  This  battery 
measures  reasoning,  spatial  visualization,  psychomotor  abilities,  and  working  memory. 

The  twelve  Army  Project  A  subtests  include  six  paper-and-pencil  spatial  tests: 
Assembling  Objects,  Maps,  Mazes,  Object  Rotation,  Orientation,  and  Figural  Reasoning; 
and  six  computerized  perceptual/psychomotor  tests:  Target  Tracking  Test  1,  Target 
Tracking  Test  2,  Target  Identification  Test-Time,  Target  Identification  Test-Hits, 
Memory  Search  Test-Time,  and  Memory  Search  Test-Hits. 
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The  Enhanced  Computer  Administration  Test  (EC AT)  battery  consists  of  nine 
tests.  Three  of  them  are  cognitive  ability  tests  that  require  computer  administration: 
Integrating  Details,  Mental  Coimters,  and  Sequential  Memory.  Three  tests  are 
psychomotor  tests  reproduced  from  the  Army  Project  A:  One-hand  Tracking,  Two-hand 
Tracking,  and  Target  Identification.  Three  of  the  tests,  Assembling  Objects,  Spatial 
Orientation,  and  Figural  Reasoning,  are  computer-administered  versions  of  Project  A 
paper-and-pencil  spatial  tests  (Wolfe,  1994). 

ASVAB  and  the  extended  Army  Project  A  subtests  scores  are  available  for  the 
Army  Project  A  while  ECAT  and  ASVAB  scores  are  available  for  the  Marine  Corps  JPM 
data.  ASVAB  scores  are  available  for  the  entire  applicant  population.  The  hands-on  job 
performance  criterion  scores  are  available  only  for  the  selected  population  and  differs 
across  jobs. 

5.2  Structural  Models 

In  the  Army  Project  A  and  the  Marine  Corps  JPM  data,  additional  tests  have  been 
similarly  used  to  extend  the  range  of  abilities  covered  by  the  ASVAB.  In  both  cases,  the 
tests  have  a  structure  similar  to  the  hierarchical  model  described  earlier.  According  to  the 
modeling  of  Gustafsson  and  Muthen  (1994),  the  most  important  factors  are  Gf  (general 
factor  interpreted  as  fluid  intelligence),  Gc  (crystallized  intelligence),  Gv/Mech  (visual 
perception  and  mechanical  knowledge).  Speed  (perceptual  speed  factor).  Math  (math 
knowledge),  and  Psymotor  (general  psychomotor  speed). 
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The  current  analyses  use  the  model  from  the  applicant  population  factor  solution 
of  Gustafsson  and  Muthen  (1994),  which  was  based  on  the  ten  ASVAB  subtests  and 
twelve  extra  variables  from  Project  A.  This  solution  was  obtained  from  an  analysis  of  a 
covariance  matrix  obtained  by  a  standard  Pearson-Lawley  adjustment  made  to  the 
covariance  matrix  for  the  sample  of  all  nine  Army  Project  A  jobs  (Gustafsson  &  Muthen, 
1994;  Muthen  &  Gustafsson,  1995).  The  same  model  is  also  applied  to  the  Marine  Corps 
JPM  data  which  includes  the  ten  ASVAB  subtests  and  eight  ECAT  variables. 

Two  models,  the  single-factor  model  and  the  Gustafsson-Muthen  hierarchical 
latent  variable  model,  are  used  as  the  differential  selection  method  for  this  study.  For  the 
single-factor  model,  there  is  only  a  general  factor  (gj)  which  influences  the  performance 
of  examinees  on  all  tests.  The  hierarchical  model  of  abilities  contains  both  general  and 
specific  factors.  The  factor  structure  for  ASVAB  and  Project  A  subtests  was  found  to  be 
very  close  to  that  of  the  factor  structure  for  ASVAB  and  ECAT  subtests  (Gustafsson  & 
Muthen,  1994).  Tables  A1  and  A2  (in  Appendix  A)  show  the  factor  structures  for  the 
Army  Project  A  and  the  Marine  Corps  JPM  datasets,  respectively. 

The  hands-on  job  performance  variable  is  regressed  on  the  six  factors  defined  in 
the  Gustafsson-Muthen  model  (1994).  Based  on  the  missing  data  theory  and  the  Pearson- 
Lawley  adjustment,  the  regression  slopes  of  the  criterion  on  the  factors  are  estimated  by 
the  regression  method  using  the  job  samples.  The  factor  scores  are  ealculated  for  each 
individual  as  the  predictors  of  job  performance.  Because  the  factor  score  coefficients  are 
estimated  from  the  applicant  population  factor  model,  the  seleetive  nature  of  the  job 
samples  is  also  considered. 
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For  each  job  sample,  both  the  Gustafsson-Muthen  model  and  the  single-factor 
model  are  obtained  so  we  can  compare  the  predictive  validities  across  jobs.  Furthermore, 
the  predicted  criterion  scores  can  be  estimated  by  these  two  different  approaches.  This 
report  investigates  whether  the  specific  factors  make  a  greater  difference  in  predicting 
hands-on  performance  than  the  general  factor  does  for  each  job. 

5.3  Procedure  of  Analyses 

Based  on  the  strategy  of  the  artificial  data  analyses  and  the  available  data  patterns, 
the  following  analysis  procedure  is  carried  out: 

Step  1 .  Obtain  the  measurement  models  based  on  the  Gustafsson-Muthen  model 
and  the  single-factor  model; 

Step 2.  Estimate  factor  scores  {g  +  5s  vs.  gj)  using  the  regression  method 
(Lawley  &  Maxwell,  1971); 

Step  3 .  Estimate  the  factor  score  determinacies; 

Step  4.  Specify  the  cutoff  score  (the  top  50  %)  based  on  gj  for  obtaining  a 
selected  sample 

Step  5.  Estimate  the  regression  slopes  (JSg  and  for  general  and  specific 
factors  with  the  hands-on  job  performance  (y)  using  the  selected  sample  (iVy); 

Step  6.  Estimate  the  predicted  hands-on  job  performance  (j))  using  the 
estimated  factor  scores  and  the  estimated  regression  coefficients; 

Step  7.  Obtain  a  selected  sample  based  on  y ; 
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Step  8.  Classify  the  true  successful  subjects  based  on  the  observed  hands-on  job 
performance  (y); 

Step  9.  Compare  ranJk  correlation  between  the  observed  hands-on  job 
performance  (y)  and  the  predicted  hands-on  job  performance  (y); 

Step  10.  Perform  sensitivity  and  specificity  analyses  to  compare  the  results  of 
different  prediction  methods. 

In  order  to  obtain  the  method  corresponding  to  the  simulation  analyses,  the  top 
50%  of  each  job  sample  based  on  the  observed  hands-on  job  performance  are  viewed  as 
true  successful  candidates  in  Figure  2  terms.  Because  of  this,  the  analyses  of  sensitivity, 
specificity,  and  proportion  of  correct  decisions  can  be  carried  out. 

As  will  be  seen,  specific  factors  are  not  all  significant  for  all  1 1  of  the  jobs.  An 
alternative  way  of  estimating  the  predicted  criterion  scores  is  to  use  the  Gf  factor  in 
addition  to  the  significant  specific  factors  as  predictors.  The  notation  of  ^  +  5^  means 
that  the  predictors  contain  the  Gf  factor  and  the  5  specific  factors  while  the  notation  of 

^  -t-  j  means  that  the  predictors  include  the  Gf  factor  and  the  specific  factors  which  are 
significant.  Because  the  regression  equations  differ  across  the  1 1  jobs,  the  approach  of 

g  +  5’'  has  a  diverse  combination  of  predictors  for  each  job.  The  notation  of  gj  denotes 
that  the  general  factor  defined  in  the  single-factor  model  is  the  only  predictor.  Thus, 
there  are  three  selection  methods  for  classifying  individuals. 
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5.4  Results 


Descriptive  statistics  for  the  estimated  factor  scores  and  the  hands-on  job 
performance  scores  are  shown  in  Table  11.  Table  12  shows  the  estimates  of  the 
standardized  prediction  equation  and  the  adjusted  R  square  for  each  job  sample.  Table  13 
displays  the  estimates  of  the  standardized  prediction  equation  for  the  top  50%  of  each  job 
sample.  The  results  of  the  Spearman  rank-order  correlation  are  shown  in  Table  14.  The 
classifications  in  different  combinations  of  selection  methods  and  job  samples  are 
summarized  in  Table  15. 

5. 4. 1  Factor  Determinacv 

Determinacies  for  the  hierarchical  factor  model  are  shown  in'  Table  11.  In  the 
Army  Project  A  data,  the  determinacies  for  Gf,  Gc,  Gv/Mech,  Speed,  Math,  and 
Psymotor  are  0.927,  0.900,  0.865,  0.840,  0.776,  and  0.634,  respectively.  In  the  Marine 
Corps  JPM  Autorhotive  Mechanic  data,  the  determinacies  for  Gf,  Gc,:  Gv/Mech,  Speed, 
Math,  and  Psymotor  are  0.935,  0.887,  0.858,  0.853.  0.747,  and  0.883,  respectively.  In  the 
Marine  Corps  JPM  Helicopter  Mechanic  data,  the  determinacies  for  Gf,  Gc,  Gv/Mech, 
Speed,  Math,  and  Psymotor  are  0.946,  0.902,  0.861,  0.859,  0.742,  and  0.916, 
respectively. 

In  summary,  the  determinacy  for  Gf  is  quite  robust  but  the  determinacy  for  Math 
drops  below  0.8  among  the  Army  Project  A  and  the  Marine  Corps  JPM.  It  is  important  to 
note  that  the  determinacy  for  Psymotor  is  only  0.634  in  the  Army  Project  A  while  the 
determinacy  for  Psymotor  is  much  higher  in  the  Marine  Corps  JPM.  The  results  show 
that  the  ECAT  subtests  tend  to  give  a  more  accurate  measure  of  Psymotor  than  the  Army 
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Project  A  subtests,  which  do  not  capture  Psymotor  very  well  in  terms  of  determinacy.  In 
general,  the  instruments  for  measuring  math  knowledge  do  not  perform  as  well  as  those 
for  measuring  other  abilities. 

5.4.2  Standardized  Resression  Equation  and  the  Adjusted  R  Square 

The  job  sample  (A^  and  the  top  50%  of  the  job  sample  (A/^)  are  used  for 
estimating  the  regression  equations.  Table  12  shows  the  standardized  estimates  for  the 
prediction  equation  with  the  total  job  sample.  In  this  table,  the  adjusted  column 
represents  the  adjusted  value  using  the  six  factors  as  predictors  across  the  1 1  jobs.  It  is 
convenient  to  compare  the  predictive  strength  and  the  regression  slopes  across  jobs  with 
respect  to  the  adjusted  value  and  the  standardized  regression  equation.  The 
hierarchical  model  shows  that  the  Gf  factor  has  a  strong  influence  on  criterion  for  all  1 1 
jobs.  The  Gc  factor  is  important  for  Radio  Operator,  Medical,  and  Automotive 
Mechanic.  The  Gv/Mech  factor  has  a  significant  effect  on  hands-on  job  performance  for 
all  jobs  except  Administrative.  The  Speed  factor  is  only  important  for  Radio  Operator 
and  Vehicle  Mechanic.  The  Math  factor  and  the  Psymotor  factor  are  significant  for 
Military  Police  and  Vehicle  Mechanic,  respectively.  The  results  of  the  adjusted  R^  show 
that  Cannon  Crewman  has  the  lowest  value.  The  highest  value  of  R^  is  about  0.57  for 
Automotive  Mechanic.  It  is  observed  that  the  same  structure  of  cognitive  ability  gives 
different  strengths  of  prediction  in  different  job  performances.  The  Marine  jobs  show  the 
highest  predictive  strength. 

The  estimates  of  the  standardized  prediction  equation  for  the  top  50%  of  each  job 
sample  are  shown  in  Table  13.  The  selected  sample  was  chosen  from  the  top  50%  of 
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ranking  by  gj.  The  results  show  that  the  Gf  factor  is  not  important  for  Cannon 
Crewman,  Tank  Crewman,  Radio  Operator,  and  Vehicle  Mechanic  in  the  selected  sample. 
The  Gc  factor  has  a  significant  effect  on  Radio  Operator,  Motor  Transport,  and 
Automotive  Mechanic.  It  is  of  interest  to  note  that  Gc  is  important  for  the  top  50%  of 
Motor  Transport  sample,  but  Gc  is  not  an  important  predictor  for  the  Motor  Transport 
sample.  The  Speed  factor  is  important  for  Vehicle  Mechanic  for  the  highest-ranked 
selectees.  In  these  selected  samples,  the  Gv/Mech  factor  becomes  more  important  than 

.•  V 

the  Gf  factor  for  Cannon  Crewman,  Tank  Crewman,  Vehicle  Mechanic,  Motor  Transport, 
and  Automotive  Mechanic.  The  significant  effect  on  Gv/Mech  is  found  in  eight  jobs. 
Math  factor  is  significant  for  Military  Police,  but  Psymotor  factor  has  no  significant 
effect  on  any  of  the  jobs.  None  of  the  six  factors  are  important  for  predicting  the  top  50% 

of  Cannon  Crewman,  so  the  alternative  method  for  g  + 1  is  to  use  the  Gf  factor  only. 
These  estimated  regression  coefficients  from  the  selected  job  samples  are  used  for  the 
following  analyses  (Spearman  rank-order  correlation  and  classification). 

5.4.3  The  Spearman  Rank-Order  Correlation  Coefficient 

The  results  of  the  Spearman  rank-order  correlations  between  the  predicted 
criterion  scores  and  the  observed  hands-on  job  performance  are  estimated  by  using  the 
three  methods  defined  above  (see  section  5.3).  A  higher  value  of  the  coefficient  means 
the  rank-ordered  y  (predicted  criterion)  based  on  a  certain  method  is  closer  to  the  ranking 
by  the  observed  hands-on  job  performance.  As  shown  in  Table  13,  the  method  of  using 

g  +  55  performs  better  than  the  methods  of  using  g  +  s*  and  gj.  Most  of  the  rank 
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correlations  between  the  observed  criterion  and  the  predicted  criterion  by  g  +  s  are 
higher  than  those  using  only  the  general  factor  (gj)  except  for  the  jobs  of  Cannon 

Crewman,  Radio  Operator,  and  Medical.  Two  sets  of  the  coefficients  for  ^  +  5  and  g] 
are  nearly  identical  for  each  of  these  three  jobs.  Therefore,  the  general  ability  is  an 
important  indicator  for  predicting  the  hands-on  performance  in  Cannon  Crewman,  Radio 
Operator,  and  Medical  jobs. 

It  is  of  interest  to  note  that  there  is  no  significant  factor  found  in  the  Cannon 

Crewman  prediction  (see  Table  13).  Thus,  the  comparison  of  g  +  s*  and  gj  can  be  seen 
as  the  comparison  of  the  Gf  factor  and  the  general  factor  for  Cannon  Crewman.  The 
results  indicate  that  for  this  jobs,  the  Gf  factor  is  similar  to  the  general  factor  in  predicting 
the  hands-on  performance.  The  results  of  Cannon  Crewman  and  Medical  demonstrate 
that  in  practice,  the  situations  with  varied  prediction  strength  (R^)  are  still  in  favor  of  the 
general  ability  as  the  predictor,  so  the  difference  among  the  three  rank  coefficients  is  tiny. 

The  results  for  the  two  Marine  jobs  show  another  pattern:  the  specific  factors 
enhance  the  precision  of  ranking  individuals  dramatically.  This  is  to  be  expected  given 
that  the  specific  factors  strengthen  the  prediction.  We  also  note  that  the  value  of  R^  is 
also  high  in  Marine  jobs.  As  the  results  of  Monte  Carlo  study  indicate  that  the  high  value 
of  R  tends  to  have  high  value  of  the  Spearman  rank-order  correlation. 

5.4.4  Sensitivity,  Specificity,  and  Proportion  of  Correct  DecMom 

Table  15  displays  the  job  classifications  with  respect  to  the  sensitivity,  specificity, 
and  the  proportion  of  correct  decisions.  When  the  value  of  R^  is  taken  into  account,  it  is 
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observed  that  the  cases  with  higher  values  of  perform  consistently  with  the  results 
from  the  simulation  analyses.  For  example,  in  Automotive  Mechanic  with  an  value  of 

0.57,  g-^Ss  performs  better  than  g-hs  and  gj  in  terms  of  the  values  of  sensitivity, 
specificity,  and  the  proportion  of  correct  decisions.  The  difference  value  is  about  4.32%. 

It  is  important  to  note  that  the  regression  coefficients  shovm  in  Table  13  are  from 
the  top  50%  of  the  job  sample.  For  Cannon  Crewman,  the  selection  method  of  g 
represents  the  Gf  factor  without  any  specific  factors  because  there  is  not  any  significant 
factor  found  in  this  regression.  The  classification  of  this  job  shows  that  general  ability  is 
the  best  predictor.  It  is  also  observed  that  including  the  predictors  of  the  specific  factors 
hamper  the  precision  of  classifications.  The  extreme  results  for  Gannon  Crewman  show 
that  g  +  55  performs  worst.  Because  of  the  lack  of  significant  predictors  and  the  low 
value  of  R  square,  the  results  for  Cannon  Crewman  are  regarded  with  suspicion. 

The  comparison  between  g  +  55  and  gj  shows  that  the  general  factor  gives  better 
predictive  classification  in  Infantryman,  Cannon  Crewman,  Tank  Crewman,  and  Radio 
Operator.  Results  from  the  rest  of  the  jobs  shed  light  on  the  use  of  specific  factors.  It  is 
expected  that  adding  the  significant  factors  to  Gf  will  lead  to  a  similar  result  obtained 
from  the  six  factors,  and  perform  better  than  the  general  ability  does.  No  certain  pattern 
is  found  in  this  real  data  application,  so  this  assumption  has  not  been  validated. 
However,  it  is  observed  that  the  results  of  sensitivity,  specificity,  and  proportion  of 

correct  decisions  are  almost  identical  for  i  +  55  and  g  +  5*  in  Helicopter  Mechanic  and 
Automotive  Mechanic  jobs. 
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Although  the  R  value  for  Vehicle  Mechanic  is  only  0.16,  the  gain  of  sensitivity 
from  g  +  55  is  about  3%.  In  practice,  the  increase  of  3%  is  a  remarkable  improvement 
for  the  selection.  Assume  a  selection  setting  with  10,000  applicants  and  50%  selection 
ratio.  Here,  an  increase  of  3%  for  sensitivity  implies  that  about  150  additional  successful 
subjects  out  of  the  5,000  successful  applicants  would  be  accepted  by  the  method  of  using 
specific  factors  in  addition  to  the  general  factor.  The  other  findings  show  that  the  • 
classification  of  Tank  Crewman  and  Radio  Operator  is  best  predicted  by  the  general 
ability.  These  results  imply  that  the  specific  skills  is  not  required  for  Tank  Crewman  and 
Radio  Operator. 

5.4.5  Profile  Description 

A  description  of  the  profile  for  the  top  1 00  individuals  in  the  observed  hands-on 
performance  for  each  job  is  given  in  the  appendix  B.  The  variable  TOP  represents  the 
rank-ordered  hands-on  performance;  when  the  subject  is  in  the  top  100  rank-ordered 
category  then  TOP  will  be  coded  to  1,  otherwise  TOP  will  be  assigned  to  0.  NEW,  SIG, 

and  GEN  variables  represent  that  the  methods  of  g  +  Ji,  g  +  5*,  and  gi  are  used  for 
predicting  hands-on  performance,  respectively.  These  three  variables  are  sorted  from 
highest  to  lowest  and  given  rank-ordered  values  for  each  subject;  then,  each  variable  is 
classified  into  different  groups  (e.g.,  group  1  includes  the  rank-order  values  from  1  to 
100;  group  2  ranging  from  101  to  200;  group  3  ranging  from  301  to  400;  group  4  ranging 
from  401  to  500;  group  5  ranging  from  501  to  600;  group  6  ranging  from  601  to  700).  A 
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two-way  table  shows  both  the  frequency  and  percentage  crosstabs  for  two  ordered 
variables  (TOP  by  NEW,  TOP  by  SIG,  or  TOP  by  GEN)  for  each  job  sample. 

Looking  at  the  result  for  job  6  (Motor  Transport),  for  example,  the  highest  rank- 
order  individuals  (TOP  =  1)  are  classified  by  three  approaches:  NEW  (g-i-55),  SIG 

/s  *  /V  ‘ 

(g  + 5  ) ,  and  GEN  (g/),  separately.  For  the  g  +  5s  method,  43  subjects  are  classified 
into  group  1,  23  subjects  are  ranked  in  group  2,  16  subjects  are  in  group  3,  12  subjects  are 

in  group  4,  and  6  subjects  are  in  group  5.  For  the  g  -i- j  approach,  41  individuals  are 
classified  into  group  1.  Based  on  the  classification  of  gy,  there  are  35  subjects  in  group 

1.  If  there  are  100  vacancies  in  Motor  Transport  job  sample,  g  +  5s  and  g  +  i*  will 
result  in  more  accurate  decisions  than  gy .  Similarly,  5.7  subjects  having  low  performance 
are  selected  by  g  +  ,  59  low-performance  subjects  are  predicted  to  be  successful  by 

^  ^  J(c 

g  +  s  ,  and  65  subjects  are  misclassified  as  potential  candidates  by  gy.  It  is  observed 
that  the  selection  method  of  using  general  and  specific  factors  tends  to  accept  more 
subjects  who  will  be  successful  and  tends  to  avoid  the  error  of  hiring  subjects  who  will 
fail  in  future  performance. 
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CHAPTER  6 


DISCUSSION 

6. 1  Discussion  of  the  Monte  Carlo  Study 

Results  from  the  simulation  study  suggest  that  using  specific  factors  in  addition  to 
a  general  factor  as  predictors  gives  better  selection  decisions.  Because  the  true  criterion 
is  available  for  each  applicant,  the  classification  of  applicants  based  on  the  true  criterion 
and  cutoff  (selection  ratio)  is  used  for  evaluating  the  success  rate  of  the  classifications  on 
these  two  selection  methods. 

It  should  be  pointed  out  that  these  findings,  as  well  as  the  findings  of  Hsu  (1995), 
support  the  notion  that  sample  size  for  selected  subjects  does  not  affect  the  prediction. 
The  results  of  sensitivity  indicate  that  the  specific  factors  play  an  important  role  in  the 
context  of  prediction  and  selection.  It  should  also  be  noted  that  the  percentage  of 
sensitivity  increase  rate  multiplied  by  the  number  of  successful  subjects  gives  information 
about  the  number  of  acquired  successful  subjects  by  the  new  selection  approach.  Figure 
19  shows  how  a  4%  increase  in  the  rate  of  sensitivity  affects  the  decision  of  accepting 
successful  applicants.  When  the  sample  size  of  applicants  is  100,000  with  a  50% 
selection  ratio,  4%  increase  in  the  sensitivity  will  lead  to  an  increase  of  accepting  2,000 
successful  subjects.  There  are  about  650,000  applicants  taking  the  ASVAB  for  military 
enlistment  every  year.  This  means  that  a  4%  of  the  increase  in  sensitivity  with  a  50% 
selection  ratio  will  increase  the  correct  selection  of  13,000  successful  subjects.  The 
results  of  sensitivity  also  show  that  the  most  selective  situation  in  terms  of  low  selection 
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ratio  (10%)  results  in  about  twice  the  increment  in  the  sensitivity  as  compared  to  the  high 
50%  selection  ratio.  This  implies  that  the  method  of  using  specifie  factors  has  potential 
contributions,  especially  in  the  setting  with  a  large  applicant  sample. 

This  report  has  examined  the  effects  of  R^,  regression  slopes,  and  factor 
determinacies  in  predictive  accuracy  in  regard  to  identification  of  applicants’  future 
performance.  It  is  clear  that  the  hierarchical  model  infhe  condition  of  high  R^,  with  more 
emphasis  on  the  general  factor  and  high  determinacies  for  specific  factors,  results  in  more 
accurate  identification  of  applicants  than  the  other  contrast  models  when  the  determinacy 
for  the  general  factor  is  held  constant.  Another  important  finding  is  that  the  model  with 
high  R  ,  high  regression  slope  on  specific  factors,  and  high  determinacies  for  specific 
factors  leads  to  the  greatest  increase  in  value  on  sensitivity  when  compared  to  the  single 
factor  model.  This  finding  is  verified  in  the  real  data  application  which  presents  evidence 
for  classification  of  Helicopter  Mechanic  and  Automotive  Mechanic  jobs. 

6.2  Discussion  of  the  Real  Data  Application 

There  are  some  restrictions  in  the  analyses  of  the  real  data  application.  First,  the 
subtest  scores  are  limited  to  part  of  the  selected  sample  so  the  classification  is  not  cairried 
out  in  the  non-selected  sample,  and  the  evaluating  methods  are  carried  out  for  the  selected 
sample.  Second,  cross-validation  is  not  carried  out  in  real  data  analyses  because  of 
sample  size. 

The  results  show  that  for  the  Army  Project  A  data,  the  Gf,  Gc,  Gv/Mech,  and 
Speed  factor  determinacies  are  quite  high,  which  implies  that  the  factor  scores  are 
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reliable.  In  the  Marine  Corps  JPM  data,  the  Math  factor  determinacy  is  about  0.75  but 
the  rest  of  the  factor  scores  are  quite  reliable.  Based  on  the  values  of  determinacies,  we 
can  conclude  that  the  EC  AT  provides  better  measure  for  Psymotor;  in  contrast,  Math 
ability  is  not  well  defined  by  the  ASVAB,  ECAT,  and  Army  Project  A  subtests.  These 
results  suggest  that  more  investigation  and  development  of  subtests  for  measuring  Math 
ability  are  needed.  ; 

Among  Vehicle  Mechanic,  Motor  Transport,  Administrative,  Helicopter 
Mechanic,  and  Automotive  Mechanic  jobs,  the  specific  factors  do  increase  the  rank 
coefficient.  The  increment  of  Spearman  rank  order  correlation  coefficient  implies  that 
these  jobs  may  require  more  specific  abilities.  For  the  prediction  of  Cannon  Crewman, 
the  general  factor  gives  slightly  better  performance  than  does  the  Gf  factor  in  addition  to 
the  Gv/Mech  factor.  In  the  case  of  Cannon  Crewman,  the  results  show  that  the  six  factors 
only  explain  5%  of  the  variance  of  the  job  performance.  The  regression  gives  a  bad  fit,  so 
any  conclusion  about  this  job  is  limited.  It  is  most  likely  that  adding  noncognitive 
predictors  to  the  ability  would  improve  the  prediction  of  hands-on  performance  for 
Cannon  Crewman. 

For  Radio  Operator,  the  regression  equation  shows  that  the  Gf  factor,  the  Gc 
factor,  the  Gv/Mech  factor,  and  the  Speed  factor  are  important  predictors.  For  the 
Medical  job,  the  regression  equation  shows  that  three  factors,  the  Gf  factor,  the  Gc  factor, 
and  the  Gv/Mech  factor,  are  the  most  important  predictors.  The  Spearman  rank-order 
coefficients,  however,  show  that  there  is  not  much  difference  among  the  three  sets  of 
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values  in  Radio  Operator  and  Medical  jobs.  It  is  observed  that  the  additional  specific 
factors  do  not  enhance  the  predictive  validity  for  these  two  jobs. 

Also,  fluid  and  crystallized  ability,  as  measured  by  intelligence  or  aptitude  tests, 
play  an  important  part  in  academic  and  occupational  success.  This  fact  demonstrates  that 
these  abilities  are  general  and  are  not  specific  to  the  tests  themselves.  The  hands-on 
performance  of  the  Administrative  job  is  better  predicted  by  the  fluid  ability  than  by  the 
general  ability.  This  points  to  the  advantage  of  using  the  hierarchical  model  for 
differentiating  the  structure  of  ability.  The  prediction  equation  of  Vehicle  Mechanic 
indicates  that  the  Gv/Mech  factor  is  more  important  than  the  Gf  factor.  This  fact 

validates  that  the  scenario  pg*  <  ,  created  in  the  simulation,  is  close  to  the  practical 

situation. 

For  the  Marine  jobs,  the  Gv/Mech  factor  for  Helicopter  Mechanic  and  Automotive 
Mechanic  and  the  Gc  factor  for  Automotive  Mechanic  significantly  improve  the 
Spearman  rank-order  correlations.  The  differences  of  the  rank  correlations  between 

g  +  (or  g  +  s  )  and  gj  are  about  0.1.  The  comparison  between  these  two  approaches 
with  a  50%  selection  ratio  shows  that  there  is  an  increase  of  about  4%  for  sensitivity, 
specificity,  and  proportion  of  correct  decisions.  It  is  of  interest  to  note  that  for 
Automotive  Mechanic  the  standardized  regression  slope  for  the  Gv/Mech  factor  (0.355)  is 
close  to  that  for  the  Gf  factor  (0.343),  while  the  standardized  regression  slope  for  the  Gc 
factor  (0.135)  is  smaller  than  that  for  the  Gf  factor.  For  Helicopter  Mechanic,  the 
standardized  regression  equation  shows  that  the  regression  slope  for  the  Gf  factor  (0.404) 
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is  much  higher  than  those  for  the  specific  factors  (e.g.,  0.295  for  Gv/Mech).  These  results 
provide  evidence  that  specific  factors  have  valuable  contributions  in  selection  and 
prediction. 

In  summary,  most  results  of  the  real  data  analyses  agree  with  the  results  of  the 
simulation  study.  The  results  suggest  that  the  Army  and  the  Marine  Corps  cart  improve 
the  prediction  of  job  performance  by  adding  speeific  factor  scores  as  predietors. 

6.3  Limitations,  Implications,  and  Recommendations 

The  results  of  the  real  data  application  point  to  the  limitations  of  the  Monte  Carlo 
study.  The  values  of  R  square  are  set  at  0.4  and  0.6  in  the  Monte  Carlo  study.  In ' 
practice,  the  values  of  R  square  for  Infantryman,  Tank  Crewman,  and  Vehicle  Mechanic 
are  below  0.2.  For  Cannon  Crewman  the  R  square  is  only  0.05.  It  would  be  useful  to 
study  models  with  mueh  lower  R  square  value  than  in  the  Monte  Carlo  study  to  examine 
the  effects  of  selection  methods.  Furthermore,  Short  (1990)  mentioned  that  the  portion  of 
items  influenced  by  specific  factors  was  found  to  be  more  influential  in  obtaining  reliable 
factor  scores  than  the  number  of  items  overall,  so  the  number  of  subtests  for  each  factor 
should  be  considered  with  respect  to  the  issue  of  factor  determinacy.  The  number  of 
subtests  for  each  specific  factor  is  designed  to  be  constant  in  the  Monte  Carlo  study. 
Since  unequal  number  of  subtests  for  each  specific  factor  could  affect  the  reliability  of 
factor  score,  it  could  be  included  in  future  studies. 

It  is  important  to  recognize  that  the  scientific  and  practical  utility  of  criterion 
validation  depends  as  much  on  the  measurement  of  the  criterion  as  it  does  on  the  quality 
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of  the  measuring  instrument  itself.  Thus,  in  many  different  types  of  training  programs, 
much  effort  and  expense  goes  into  the  development  of  a  test  for  predicting  who  will 
benefit  from  the  program  in  terms  of  subsequent  job  performance.  In'  the  area  of  special 
education  or  prevention  programs,  the  use  of  factor  scores  will  lead  to  more  accurate 
identification  and  classification;  for  students  regarding  future  performance.  Because  of 
the  limited  resources,  how  to  determine  which  smdent  receives  treatment  and  which 
student  is  ineligible  for  treatment  represents  an  important  issue.  For  example,  the 
traditional  approaches  to  the  identification  of  reading  problems  require  comparison  of 
standardized  achievement  and  IQ  measures.  Factor  scores  can  be  applied  to 
differentiating  skill  areas  (e.g.,  general  reading  ability,  spelling  skill,  and  vocabulary 
skill).  When  evaluating  these  differences  in  the  light  of  the  theory  of  reading  problems, 
the  profiles  of  factor  scores  will  lead  to  the  prediction  that  individuals  with  less  spelling 
skill  tend  to  be  the  risk  group  for  poor  academic  outcome.  The  use  of  specific  factors  can 
be  applied  to  the  specialized  training  program.  For  instance,  the  program  for  musical 
talents  will  admit  subjects  with  musical  aptitude  (e.g.,  special  talent  in  pitch,  rhythm,  or 
tone)  rather  than  those  with  high  academic  achievement  or  IQ  score. 

Predictive  validity  has  been  used  in  the  fields  of  psychology  and  education  mainly 
for  analyzing  the  validity  of  certain  types  of  tests  and  selection  procedures.  Many 
standardized  intelligence  tests,  achievement  tests,  and  ability  tests  were  designed  to 
provide  information  for  the  selection  and  placement  of  students  and  for  comparing 
students,  schools,  and  school  districts  with  one  another.  Since  many  children  have  been 
misclassified  by  standardized  tests  and  on  this  basis  have  been  assigned  to  programs  with 
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minimal  content,  low  expectations,  and  restricted  (rather  than  enriched)  teaching 
approaches,  the  link  between  assessment  and  intervention  becomes  more  important  than 
the  issues  of  placement  and  prediction.  A  new  type  of  test  should  provide  diagnostic 
information  about  students’  preconceptions,  learning  strategies,  and  metacognitive  and 
affective  thought  processes.  The  approach  of  using  factor  scores  should  contribute  to 
effective  diagnostic  ^alysis.  ^ 

The  most  difficult  problem  in  social  science  studies  is  that  the  hypothetical 
concepts  and  constructs  are  not  directly  measurable.  Although  such  concepts  and 
constructs,  or  latent  variables,  cannot  be  directly  measured,  a  number  of  variables  can  be 
used  to  measure  various  aspects  of  these  latent  variables  more  or  less  accurately.  We 
may  regard  the  observed  variables  as  indicators  of  the  latent  variables.  Each  indicator  has 
a  relationship  with  the  latent  variable,  but  if  we  take  one  indicator  alone  to  measure  the 
latent  variable,  we  would  obtain  a  biased  measurement.  Using  several  indicators  of  each 
latent  variable  gives  a  better  measurement  of  the  latent  variable.  Another  reason  for  using 
latent  variables  in  behavioral  and  socioeconomic  studies  is  that  most  of  the  measurements 
employed  contain  sizable  errors  of  measurement,  which,  if  not  taken  into  account,  can 
cause  severe  bias  in  the  results.  Errors  of  measurement  arise  because  of  imperfection  in 
the  various  measurement  instruments  that  are  used  to  measure  people’s  behavior, 
attitudes,  feelings,  and  motivations.  Even  if  we  could  construct  valid  measurement  for 
these  traits,  it  is  usually  impossible  to  obtain  perfectly  reliable  variables.  In  practice, 
using  factor  scores  instead  of  raw  score  or  composite  scores  will  be  proper  for  profile 
analysis. 
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As  an  alternative  to  dollar  value  units,  utility  gains  can  be  expressed  as  percentage 
increases  in  output.  It  is  easy  to  realize  that  such  output  increases  imply  large  dollar 
values,  and  there  is  a  tendency  to  regard  dollar  estimates  as  indicating  greater  utility  than 
the  percentage-increase  estimates.  It  should  be  emphasized  that  the  use  of  specific  factor 
scores  deserves  special  recognition  as  an  improvement  in  selection  and  prediction.  Given 
the  findings,  these;  results  suggest  that  the  specific  factor  scores  may  facilitate  the 
identification  of  students  with  reading  disabilities  and  learning  impairments  in  special 

V 

populations. 

In  the  Army  Project  A  data  and  the  Marine  Corps  JPM  data,  each  job  has  different 
hands-on  job  performance  so  the  use  of  factor  score  can  obtain  unbiased  estimates  of 
regression  slopes  for  each  job.  These  equations  can  then  be  applied  to  the  new  enlistment 
cohort  for  selection  and  classification  reference.  Moreover,  based  on  the  different 
prediction  equations,  the  different  predicted  hands-on  performance  for  each  of  the 
different  jobs  could  be  computed  for  each  individual.  The  information  of  predicted 
hands-on  performance  could  be  used  for  matching  people  to  jobs. 

Previous  research  (Campbell,  McHenry,  &  Wise,  1990)  using  the  Army  Project  A 
data  showed  that  some  dimensions  of  job  behavior,  such  as  physical  fitness  and  military 
bearing,  are  better  predicted  by  noncognitive  than  cognitive  predictors,  and  are  better 
predicted  by  some  noncognitive  predictors  than  by  others.  Hogan  (1991),  Tett  et  al. 
(1991),  and  Schmit  et  al.  (1995)  suggested  that  some  dimensions  of  job  behavior  can  be 
predicted  reliably  by  personality  measures.  Although  the  use  of  personality  as  a  predictor 
in  personnel  selection  has  not  been  substantially  successful  in  the  past,  Irving  (1993)  has 
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suggested  that  personality  measures  are  related  to  performance  criteria  which  are 
unrelated  to  cognitive  ability  when  the  traits  measured  are  conceptually  related  to  these 
criteria.  It  seems  that  personality  measures  may  predict  job  performance  dimensions 
which  cannot  be  predicted  by  cognitive  ability  measures.  The  use  of  personality 
measures  in  persoruiel  selection  may  be  warranted  when  a  careful  job  analysis  is 
undertal^n  to  determine  which  performance  dimensions  may  be  related  to  personality 
traits. 

As  a  final  comment,  the  results  of  this  report  should  be  viewed  with  caution. 
Only  a  limited  number  of  situations  are  included  in  the  Monte  Carlo  study.  Further 
research  using  data  on  additional  occupations  is  necessary  to  corroborate  the  findings 
presented  here.  Despite  its  limitations,  this  report  has  shed  some  light  on  the  nature  of 
the  relationship  between  aptitude  tests  and  job  performance  and  may  stimulate  additional 
research  on  an  important  topic  in  the  area  of  prediction.  In  summary,  the  results  indicate 
that  the  method  of  utilizing  specific  factor  information  studied  in  this  research  is  valuable 
for  personnel  selection. 
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Table  1 

Design  of  the  Simulation  Study 


1\ 


Note:  Case  represents  ^  +  1  5,  g  +  2  and  g  +  3 

Factor  represents  p*,  and  A  s- 


Table  2 

Model  Parameter  Values  (g  +  1  s) 
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Table  2 

Model  Parameter  Values  {g  +  2  s ) 
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Regression  Slope 


Table  2 

Model  Parameter  Values  (g  +3  s) 
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Factor  Detemiinacy  0.91  0.62  0.72  0.61  0.90  0  79  0.76  0.75  0.91  0.62  0.72  0.61  0.90  0.79  0.76  0.75  0.91  0.62  0.72  0.61 


Tables 

Summan'  of  Classification  and  Spearman  Rank  Correlation  for  g  +  U 

_ 

- - - - - - - — - — S - - -  i  i: 

— 

Sensitivity  Specificity  Correct  Rank  Correlation  Sensitivity  Specificity  Correct  Rank  Correlation 

N=250  R=0.5 


Model 

1 

68.75% 

68.75% 

68.75% 

0.539 

69.62% 

69.62% 

69.62% 

0.565 

2 

70.20% 

70.20% 

70.20% 

0.577 

70.59% 

70.59% 

70.59% 

0.588 

3 

65.93% 

65.93% 

65.93% 

0.462 

68.47% 

68.47% 

68.47% 

0.529 

: 

4 

‘68.13% 

68,13% 

68.13% 

0.524 

70.33% 

70.33% 

70.33% 

0.578 

5 

62.30% 

62.30% 

62.30% 

0.364 

66.56% 

66.56% 

66.56% 

0.486 

6 

65.66% 

65.66% 

65.66% 

0.447 

69.87% 

69.87% 

69.87% 

0.566 

7 

73.64% 

73.64% 

73.64% 

0.660 

75.04% 

75.04% 

75.04% 

0  690 

8 

75.61% 

75.61% 

75.61% 

0.703 

76.34% 

76.34% 

76.34% 

0.720 

9 

69.93% 

69.93% 

69.93% 

0.568 

73.47% 

73.47% 

73.47% 

0  653 

10 

73.04% 

73.04% 

73.04% 

0.640 

75.97% 

75.97% 

75.97% 

0  708 

11 

64.73% 

64.73% 

64.73% 

0.435 

71.22% 

71.22% 

71.22% 

0.599 

12 

i68.8  /“"b 

68.87% 

68.87% 

0.540 

74.70% 

■  74.70% 

74.70% 

0.686 

N=250 

R=0.1 

Model 

I 

35.45% 

92.83% 

87.09% 

0.535 

36.99% 

93.00% 

87.40% 

0  558 

2 

38.52% 

93.17% 

87.70% 

0.573 

39.31% 

93.26% 

87.86% 

0.583 

3 

30.94% 

92.33% 

86.19% 

0.466 

35.16% 

92.80% 

87.03% 

0.528 

4 

35.35% 

92.82% 

87.07% 

0.525 

38.48% 

93.16% 

87.70% 

0.573 

5 

■24.98% 

91.66% 

85.00% 

0.360 

32.52% 

92.50% 

86.50% 

0.486 

6 

29.69% 

92.19% 

85.94% 

0.439 

36.85% 

92.98% 

87.37% 

0.549 

7 

45.6  7”''o 

93.96% 

89.13% 

0.662 

47.72% 

94.19% 

89.54% 

0.692 

8 

48.41% 

94.27% 

89.68% 

0.707 

49.78% 

94.42% 

89.96% 

0.720 

9 

38.18% 

93.13% 

87.64% 

0.570 

44.74% 

93.86% 

88.95% 

0.653 

10 

43.94% 

93.77% 

88.79% 

0.645 

49.22% 

94.36% 

89.84% 

0.707 

11 

30.23% 

92.25% 

86.05% 

0.443 

40.70% 

93.41% 

88.14% 

0.600 

12 

.?6. 1 7% 

92.91% 

87.23% 

0.543 

47.51% 

94.17% 

89.50% 

0.688 

N=500 

R=0.5 

Model 

1 

68.60% 

68.60% 

68.60% 

0.535 

69.56% 

69.56% 

69.56% 

0.561 

2 

70.08% 

70.08% 

70.08% 

0.573 

70.60% 

70.60% 

70.60% 

0.586 

3 

65.97% 

65.97% 

65.97% 

0.461 

68.52% 

68.52% 

68.52% 

0.528 

4 

68.44% 

68.44% 

68.44% 

0.527 

70.56% 

70.56% 

70.56% 

0.579 

5 

62.53°b 

62.53% 

62.53% 

0.363 

67.12% 

67.12% 

67.12% 

0.493 

6 

65.18‘^^) 

65.18% 

65.18% 

0.440 

69.60% 

69.60% 

69.60% 

0.558 

7 

73.60% 

73.60% 

73.60% 

0.659 

75.12% 

75.12% 

75.12% 

0.694 

8 

75.91% 

75.91% 

75.91% 

0.709 

76.59% 

76.59% 

76.59% 

0.725 

9 

70.28% 

70.28% 

70.28% 

0.575 

73.55% 

73.55% 

73.55% 

0.657 

10 

73.12% 

73.12% 

73.12% 

0.645 

76.02% 

76.02% 

76.02% 

0.711 

1 1 

65.13% 

65.13% 

65.13% 

0.446 

71.48% 

71.48% 

71.48% 

0.610 

12 

68.86% 

68.86% 

68.86% 

0.542 

75.13% 

75.13% 

75.13% 

0.692 

S-=500 

R=0.1 

Model 

1 

35.68% 

92.85% 

87.14% 

0.538 

37.70% 

93.08% 

87.54% 

0.563 

2 

38.23% 

93.14% 

87.65% 

0.575 

39.25% 

93.25% 

87.85% 

0.586 

3 

31.54% 

92.39% 

86.31% 

0.465 

35.80% 

92.87% 

87.16% 

0.532 

4 

35.00% 

92.78% 

87.00% 

0.524 

38.42% 

93.16% 

87.68% 

0.574 

5 

25.52% 

91.72% 

85.10% 

0.362 

33.25% 

92.58% 

86.65% 

0.491 

6 

30.03% 

92.23% 

86.01% 

0.443 

37.23% 

93.03% 

87.45% 

0.555 

7 

45.23% 

93.91% 

89.05% 

0.663 

47.78% 

94.20% 

89.56% 

0.695 

8 

48.52% 

94.28% 

89.70% 

0.707 

49.72% 

94.41% 

89.94% 

0.722 

9 

37.95% 

93.11% 

87.59% 

0.571 

44.23% 

93.80% 

88.85% 

0.655 

10 

44.01% 

93.78% 

88.80% 

0.648 

49.21% 

94.36% 

89.84% 

0.711 

11 

29.86% 

92.21% 

85.97% 

0.443 

40.54% 

93.39% 

88.11% 

0.605 

12 

35.99% 

92.89% 

87.20% 

0.541 

47.57% 

94.17% 

89.51% 

0.690 

>Jote  N  represents  selected  sample  size. 
R  represents  selection  ratio. 
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Table  3 

Summarv-  of  Classification  and  Spearman  Rank  Correlation  for  g  +  1  ^ 


Difference  (pain  from  j? 

1  5  ) 

Sensitivity  Specificity  Correct  Rank  Coixciation 

N=250  R=0.5 

Model  1  0.87% 

0.87% 

0.87% 

0.026 

2 

0.39% 

0.39% 

0.39% 

0.012 

3 

2.54% 

2.54% 

2.54% 

0.067 

4 

2.20% 

2.20% 

2.20% 

0.054 

5 

4.26% 

4.26% 

4.26% 

0.122 

6 

4.21% 

4.21% 

4.21% 

0.119 

7 

1.41% 

1.41% 

1.41% 

0.031 

8 

0.73% 

0.73% 

0.73% 

0.017 

9 

3.54% 

3.54% 

3.54% 

0.085 

10 

2.92% 

2.92% 

2.92% 

0.068 

11 

6.48% 

6.48% 

6.48% 

Q.164 

12 

5.84% 

5.84% 

5.84% 

0.147 

N=250  R=0.1 

Model  1 

1.54% 

0.17% 

0.31% 

0.022 

2 

0.79% 

0.09% 

0.16% 

0.010 

3 

4.22% 

0.47% 

0.84% 

0.062 

4 

3.14% 

0.35% 

0.63% 

0.048 

5 

7.55% 

0.84% 

1.51% 

0.126 

6 

7.16% 

0.80% 

1.43% 

O.Ill 

7 

2.05% 

0.23% 

0.41% 

0.030 

8 

1.36% 

0.15% 

0.27% 

0.013 

9 

6.56% 

0.73% 

1.31% 

0.083 

10 

5.28% 

0.59% 

1.06% 

0.062 

II 

10.47% 

1.16% 

2.09% 

0.157 

12 

11.34% 

1.26% 

2.27% 

0.145 

N-500  R=0.5 

Model  1 

0.96% 

0.96% 

0.96% 

0.027 

2 

0.52% 

0.52% 

0.52% 

0.013 

3 

2.55% 

2.55% 

2.55% 

0.067 

4 

2.13% 

2.13% 

2.13% 

0.052 

5 

4.60% 

4.60% 

4.60% 

0.130 

6 

4.42% 

4.42% 

4.42% 

0.118 

7 

1.51% 

1.51% 

1.51% 

0.035 

8 

0.68% 

0.68% 

0.68% 

0.016 

9 

3.27% 

3.27% 

3.27% 

0.082 

10 

2.90% 

2.90% 

2.90% 

0.066 

11 

6.35% 

6.35% 

6.35% 

0.163 

12 

6.27% 

6.27% 

6.21% 

0.150 

N=500  R=0,1 

Model  1 

2.02% 

0.22% 

0.40% 

0.025 

2 

1.02% 

0.11% 

0.20% 

0.011 

3 

4.26% 

0.47% 

0.85% 

0.067 

4 

3.42% 

0.38% 

0.68% 

0.050 

5 

7.74% 

0.86% 

1.55% 

0.129 

6 

7.20% 

0.80% 

1.44% 

0.113 

7 

2.55% 

0.28% 

0.51% 

0.032 

8 

1.20% 

0.13% 

0.24% 

0.015 

9 

6.28% 

0.70% 

1.26% 

0.085 

10 

5.20% 

0.58% 

1.04% 

0.064 

11 

10.67% 

1.19% 

2.13% 

0.162 

12 

11.58% 

1.29% 

2.32% 

0.149 
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Table  4 

Summan'  of  Classification  and  Spearman  Rank  Correlation  for  ^  +  2  ^ 


Sensitivity  Specificity  Correct  Rank  Coirektion 


Sensitivity  •  Specificity  Correct 


N=250 

Model 


N=250 

Model 


R=0.5 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

R=0.1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 


69.13% 
70.10% 
65.65% 
66.72% 
64.61% 
66.25% 
74  38% 
74,94% 
68.48% 
71.23% 
68.56% 
70.31% 


36.90% 

38.05% 

30.87% 

33.38% 

29.21% 

31.500^1 

45.63°o 

47.88% 

37.33% 

40.57% 

35.52“'o 

38.70% 


N=500 

Model 


R=0.5 

I 


N-500 

Model 


10 


12 


R=0.1 

1 


69.08% 

70.07% 

65.36% 

66.82% 

64.72% 

66.05% 

74.4.3% 

75.44% 

69.63% 

71.43% 

68.24% 

70.49% 


69.13% 

70.10% 

65.65% 

66.72% 

64.61% 

66.25% 

74.38% 

74,94% 

68.48% 

71,23% 

68.56% 

70,31% 


92.99% 
9?.  12% 
92.32% 
02.60% 
92.14% 
92.39% 
93.96% 
94.21% 
93.04% 
93.40% 
92.84% 
93.19% 


69.08% 

70.07% 

65.36% 

66.82% 

64.72% 

66.05% 

74.43% 

75.44% 

69.63% 

71.43% 

68.24% 

70.49% 


8 

9 

10 
11 
12 


36.72% 

38.11% 

50.69% 

32.98% 

29.17% 

31.49% 

46.20% 

48.44% 

37.58% 

40.38% 

35.47% 

38.70% 


92.97% 

93.12% 

92.30% 

92.55% 

92.13% 

92.39% 

94.02% 

94,27% 

93.06% 

93.38% 

92.83% 

93.19% 


69.13% 

70.10% 

65.65% 

66.72% 

64.61% 

66.25% 

74.38% 

74.94% 

68.48% 

71.23% 

68.56% 

70.31% 


87.38% 

87.61% 

86.17% 

86.68% 

85.85% 

86.30% 

89.13% 

89.58% 

87.47% 

88.11% 

87.10% 

87.74% 


69.08% 

70.07% 

65.36% 

66.82% 

64.72% 

66.05% 

74.43% 

75.44% 

69.63% 

71.43% 

68.24% 

70.49% 


87.34% 

87.62% 

86.14% 

86.60% 

85.83% 

86.30% 

89.24% 

89.69% 

87.52% 

88.08% 

87.09% 

87.74% 


iVote  X  represents  selected  sample  s 
R  represents  selection  ratio. 


0.548 

0.570 

0.452 

0.490 

0.426 

0.471 

0.674 

0.694 

0.532 

0.605 

0.526 

0.576 


0.548 

0.568 

0.454 

0.492 

0.429 

0.472 

0.676 

0.702 

0.559 

0.604 

0.526 

0.580 


0.546 

0.567 

0.452 

0.488 

0.429 

0.467 

0.677 

0.701 

0.560 

0.606 

0.524 

0.575 


0.549 

0.568 

0.454 

0.487 

0.425 

0.471 

0.675 

0.701 

0.562 

0.605 

0.525 

0.577 


69.40% 

70.83% 

66.57% 

68.64% 

65.56% 

68.27% 

74.68% 

76.08% 

70.56% 

73.50% 

69.72% 

72,90% 


37.56% 

39.34% 

32.09% 

36,26% 

30.96% 

34.46% 

46.79% 

50.39% 

39.71% 

45.10% 

38.13% 

43.83% 


69.42% 

70.81% 

66.44% 

68.61% 

65.99% 

67.86% 

75.03% 

76.79% 

70.76% 

74.04% 

69.85% 

73.23% 


37.27% 

39.57% 

32.30% 

36.32% 

31.24% 

35.39% 

47.12% 

50.68% 

39.96% 

45.38% 

38.37% 

44.12% 


69.40% 

70.83% 

66.57% 

68.64% 

65.56% 

68.27% 

74.68% 

76.08% 

70.56% 

73.50% 

69.72% 

72.90% 


93.06% 

93.26% 

92.45% 

92.92% 

92.33% 


94.49^*0 

93.30% 

9.3.90% 

93.13% 

93.76% 


69.42% 

70.81% 

66.44% 

68.61% 

65.99% 

67.86% 

75.03% 

76.79% 

70.76% 

74.04% 

69.85% 

73.23% 


93.03% 

93.29% 

92.48% 

92.92% 

92.36% 

92.82% 

94.12% 

94.52% 

93.33% 

93.93% 

93.15% 

93.79% 


69.40% 

70.83% 

66.57% 

68.64% 

65.56% 

68.27% 

74.68% 

76.08% 

70.56% 

73.50% 

69.72% 

72.90% 


87.51% 

87.87% 

86.42% 

87.25% 

86.19% 

86.89% 

89.36% 

90.08% 

87.94% 

89.02% 

87.63% 

88.77% 


69.42% 

70.81% 

66.44% 

68.61% 

65.99% 

67.86% 

75.03% 

76.79% 

70.76% 

74.04% 

69.85% 

73.23% 


87.45% 

87,91% 

86.46% 

87.26% 

86.25% 

87.08% 

89.42% 

90.14% 

87.99% 

89.08% 

87.67% 

88.82% 


Rank  Correlation 


0.556 

0.591 

0.475 

0.536 

0.453 

0.522 

0.684 

0.718 

0.564 

0.661 

0.561 

0.641 


0.553 

0.583 

0.473 

0.536 

0.454 

0.520 

0.685 

0.725 

0.591 

0.664 

0.564 

0.647 


0.557 

0.586 

0.479 

0.538 

0.462 

0.521 

0.689 

0.727 

0.592 

0.666 

0.565 

0.645 


0.557 

0.586 

0.479 

0.536 

0.458 

0.526 

0.686 

0.727 

0.596 

0.667 

0.568 

0.650 
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Table  4 

Summary  of  Classification  and  Spearman  Rank  Correlation  for  g +2  s 

- - Difference  (  gain  from  g  +  2  j  ) 

Sensitivity  Specificity  Coirect  Rank  Correlation 


N=250  R=0.5 
Model  1  0.27% 

2  0.73% 

3  0.92% 

4  1.93% 

5  0.94% 

6  2.02% 

7  0.29% 

8  1.14% 

9  2.08% 

10  2.27% 

n  1.16% 

12  2.58% 


0.27%  0.27% 

0.73%  0.73% 

0.92%  0.92% 

1.93%  1.93% 

0.94%  0.94% 

2.02%  2.02% 

0.29%  0.29% 

1.14%  l.I4o^ 

2.08%  2.08% 

2.27%  2.27% 

1.16%  1.16% 

2.58%  2.58% 


0.007 

0.020 

0.023 

0.046 

0.027 

0.052 

0.010 

0.024 

0.032 

0.056 

0.035 

0.065 


N=250  R=0.1 


Model  1 

0.66% 

0.07% 

0.13% 

0.005 

2 

1.29% 

0.14% 

0.26% 

0.015 

3 

1.22% 

0.14% 

0.24% 

0.020 

4 

2.88% 

0.32% 

0.58% 

0.044 

5 

1.69% 

0.19% 

0.34% 

0.025 

6 

2.95% 

0.33% 

0.59% 

0.048 

7 

1.16% 

0.13% 

0.23% 

0.010 

8 

2.50% 

0.28% 

0.50% 

0.024 

9 

2.38% 

0.26% 

0.48% 

0.032 

10 

4.53% 

0.50% 

0.91% 

0.059 

11 

2.61% 

0.29% 

0.52% 

0.037 

12 

N=500  R=0.5 

5.13% 

0.57% 

1.03% 

0.067 

Model 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 


0.34% 

0.74% 

1.09% 

1.80% 

1.27% 

1.81% 

0.60% 

1.35% 

1.13% 

2.62% 

1.61% 

2.75% 


0.34% 

0.74% 

1.09% 

1.80% 

1.27% 

1.81% 

0.60% 

1.35% 

1.13% 

2.62% 

1.61% 

2.75% 


0.34% 

0.74% 

1.09% 

1.80% 

1.27% 

1.81% 

0.60% 

1.35% 

1.13% 

2.62% 

1.61% 

2.75% 


0.010 

0.019 

0.027 

0.050 

0.032 

0.054 

0.012 

0.026 

0.031 

0.061 

0.041 

0.070 


N=500  R=0.1 
.Model  1  0.56% 

2  1.46% 

3  1.60% 

4  3.33% 

5  2.07% 

6  3.90% 

7  0.92% 

8  2.24% 

9  2.39% 

10  5.00% 

1 1  2.90% 

12  5.43% 


0.06%  0.11% 

0.16%  0.29% 

0.18%  0.32% 

0.37%  0.67% 

0.23%  0.41% 

0.43%  0.78% 

0.10%  0.18% 

0.25%  0.45% 

0.27%  0.48% 

0.55%  1.00% 

0.32%  0.58% 

0.60%  1.09% 


0.008 

0.019 

0.025 

0.049 

0.033 

0.055 

O.OII 

0.025 

0.034 

0.062 

0.043 

0.072 
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Table  5 

Summary  of  Classification  and  Spearman  Rank  Correlation  for  ^  +  3  ^ 


- - - - - L 

Sensitivity  Specificity  Correct 


N=250  R=0.5 
Model  1  69.09% 

2  69.72% 

3  65.73% 

4  66.73% 

5  64.30% 

6  64.64% 

7  74.22% 

8  75.02% 

9  69.76% 

10  70.59% 

11  67.20% 

12  68.17% 

N=250  R=0.I 
Model  1  36.42®/o 

2  37.76% 

3  30.81% 

4  31.74% 

5  28.27% 

6  28.64% 

7  45.72% 

8  48.14% 

9  37.40% 

10  38.74% 

11  33.94% 

12  34.47% 

N=500  R=0.5 
Model  I  69.09% 

2  69.92% 

3  65.68% 

4  66.02% 

5  64.15% 

6  64.50% 

7  74.26% 

8  75.60% 

9  69.66% 

10  70.37% 

11  67.49% 

12  68.20% 

N=500  R=0.1 
Model  1  36.35% 

2  38.24% 

3  30.75% 

4  31.49% 

5  28.20% 

6  29.02% 

7  45.74% 

8  48.30% 

9  37.67% 

10  38.88% 

11  33.43% 

12  34.97% 


69.09% 

69.09% 

69.72% 

69.72% 

65.73% 

65.73% 

66.73% 

66.73% 

64.30% 

64.30% 

64.64% 

64.64% 

74.22% 

74.22% 

75.02% 

75.02% 

69.76% 

69.76% 

70.59% 

70.59% 

67.20% 

67.20% 

68.17% 

68.17% 

92.94% 

87.28% 

93.08% 

87.55% 

92.31% 

86.16% 

92.42% 

86.35% 

92.03% 

85.65% 

92.07% 

85.73% 

93.97% 

89.14% 

94.24% 

89.63% 

93.04% 

87.48% 

93.19% 

87.75% 

92.66% 

U.19% 

92.72% 

86.89% 

69.09% 

69.09% 

69.92% 

69.92% 

65.68% 

65.68% 

66.02% 

66.02% 

64.15% 

64.15% 

64.50% 

64.50% 

74.26% 

74.26% 

75.60% 

75.60% 

69.66% 

69.66% 

70.37% 

70.37% 

67.49% 

67.49% 

68.20% 

68.20% 

92.93% 

87.27% 

93.14% 

87.65% 

92.31% 

86.15% 

92.39% 

86.30% 

92.02% 

85.64% 

92.11% 

85.80% 

93.97% 

89.15% 

94.26% 

89.66% 

93.07% 

87.53% 

93.21% 

87.78% 

92.60% 

86.69% 

92.77%  86.99% 


Xote  N  represents  selected  sample  size, 
R  represents  selection  ratio. 


Rank  Correlation 


- - - - - ^ _ S  ^  ^ _ 

Sensitivity  -  Specificity  Correct  Rank  Correlation 


69.31%  69.31% 

70.32%  70.32% 

67.36%  67.36% 

69.68%  69.68% 

66.35%  66.35% 

68.44%  68.44% 

74.66%  74.66% 

76.16%  76.16% 

71.68%  71.68% 

74.29%  74.29% 

70.67%  70.67% 

73.29%  73.29% 


0.547 

0.564 

0.458 

0.479 

0.412 

0.423 

0.670 

0.696 

0.563 

0.580 

0.503, 

0.519 


0.544 

0.566 

0.455 

0.469 

0.411 

0.423 

0.674 

0.700 

0.562 

0.578 

0.503 

0.521 


0.545 

0.566 

0.453 

0.469 

0.411 

0.426 

0.672 

0.700 

0.557 

0.582 

0.504 

0.521 


36.98% 

38.68% 

33.15% 

36.09% 

32.02% 

34.69% 

46.57% 

49.77% 

41.44% 

45.68% 

39.72% 

44.01% 


69.63% 

70.56% 

67.40% 

68.90% 

66.74% 

68.54% 

74.88% 

76.56% 

71.88% 

74.21% 

71.10% 

73.52% 


93.00% 

93.19% 

92.57% 

92.90% 

92-45% 

92.74% 

94.06% 

94.42% 

93.49% 

93.96% 

93.30% 

93.78% 


69.63% 

70.56% 

67.40% 

68.90% 

66.74% 

68.54% 

74.88% 

76.56% 

71.88% 

74.21% 

71.10% 

73.52% 


69.31% 

70.32% 

67.36% 

69.68% 

66.35% 

68.44% 

74.66% 

76.16% 

71.68% 

74.29% 

70.67% 

73.29% 


87.40% 

87.74% 

86.63% 

87.22% 

86.40% 

86.94% 

89.31% 

89.95% 

88.29% 

89.14% 

87.94% 

88.80% 


69.63% 

70.56% 

67.40% 

68.90% 

66.74% 

68.54% 

74.88% 

76.56% 

71.88% 

74.21% 

71.10% 

73.52% 


0.556 

0.582 

0.501 

0.554 

0.479 

0.529 

0.683 

0.718 

0.617 

0.673 

0.593 

0.650 


0.546 

0.576 

0.489 

0.534 

0.472 

0.520 

0.682 

0.718 

0.614 

0.668 

0.587 

0.646 


0.556 

0.584 

0.500 

0.544 

0.483 

0.532 

0.684 

0.721 

0.615 

0.675 

0.596 

0.653 


0.546 

0.566 

0.456 

0.470 

0.410 

0.423 

0.675 

0.701 

0.561 

0.580 

0.503 

0.520 


37.05%  93.01% 

39.43%  93.27% 

33.26%  92.58% 

36.28%  92.92% 

32.63%  92.51% 

35.21%  92.80% 

46.91%  94.10% 

50.15%  94.46% 

41.77%  93.53% 

46.02%  94.00% 

39.77%  93.31% 

44.39%  93.82% 


87.41%  0.553 

87.89%  0.581 

86.65%  0.497 

87.26%  0.542 

86.53%  0.480 

87.04%  0.526 

89.38%  0.686 

90.03%  0.721 

88,35%  0.617 

89.20%  0.672 

87.95%  0.593 

88.88%  0.651 
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Table  5 

Summary  of  Classification  and  Spearman  Rank  Correlation  for  g  +  3  ^ 


Difference  (  gain  from  ^  ^2  s  ) 


Sensitivity  Specificity 

Conrect 

Rank  Correlation 

N=250 

R=0.5 

Model 

1 

0.22% 

0.22% 

0-22% 

0.009 

2 

0.60% 

0.60% 

0.60% 

0.017 

3 

1.63% 

1.63% 

1.63% 

0.043 

4 

2.94% 

2.94% 

2.94% 

0.075 

5 

2.05% 

2.05% 

2.05% 

0.067 

6 

3.80% 

3.80% 

3.80% 

0.106 

7 

0.44% 

0.44% 

0.44% 

0.012 

8 

1.14% 

1.14% 

1.14% 

0.022 

9 

1.92% 

1.92% 

1.92% 

0.053 

10 

3.70% 

3.70% 

3.70% 

0.093 

11 

3.47% 

3.47% 

3.47% 

0.090 

12 

5.12% 

5.12%' 

5.12% 

0.131 

N=250 

R=0.1 

Model 

1 

0.56% 

0.06% 

0.11% 

0.002 

2 

0.92% 

0.10% 

0.18% 

0.010 

3 

2.34% 

0.26% 

0.47% 

0.034 

4 

4.34% 

0.48% 

0.87% 

0.065 

5 

3.75% 

0.42% 

0.75% 

0.061 

6 

6.05% 

0.67% 

1.21% 

0.097 

7 

0.85% 

0.09% 

0.17% 

0.009 

8 

1.64% 

0.18% 

0.33% 

0.018 

9 

4.04% 

0.45% 

0.81% 

0.052 

10 

6.94% 

0.77% 

1.39% 

0.090 

11 

5.78% 

0.64% 

1.16% 

0.084 

12 

9.54% 

1.06% 

1.91% 

0.125 

N=500 

R=0.5 

Model 

1 

0.55% 

0.55% 

0.55% 

0.011 

2 

0.64% 

0.64% 

0.64% 

0.017 

3 

1.71% 

1.71% 

1.71% 

0.047 

4 

2.88% 

2.88% 

2.88% 

0.076 

5 

2.59% 

2.59% 

2.59% 

0.072 

6 

4.04% 

4.04% 

4.04% 

0.106 

7 

0.62% 

0.62% 

0.62% 

0.012 

8 

0.96% 

0.96% 

0.96% 

0.021 

9 

2.22% 

2.22% 

2.22% 

0.058 

10 

3.84% 

3.84% 

3.84% 

0.092 

11 

3.61% 

3.61% 

3.61% 

0.091 

12 

5.32% 

5.32% 

5.32% 

0.132 

N=500 

R=0.1 

Model 

I 

0.70% 

0.08% 

0.14% 

0.007 

2 

1.20% 

0.13% 

0.24% 

0.014 

3 

2.51% 

0.28% 

0.50% 

0.041 

4 

4.79% 

0.53% 

0.96% 

0.072 

5 

4.43% 

0.49% 

0.89% 

0.070 

6 

6.18% 

0.69% 

1.24% 

0.103 

7 

1.16% 

0.13% 

0.23% 

0.011 

8 

1.85% 

0.21% 

0.37% 

0.020 

9 

4.10% 

0.46% 

0.82% 

0.056 

10 

7.14% 

0.79% 

1.43% 

0.092 

11 

6.34% 

0.70% 

1.27% 

0.090 

12 

9.42% 

1.05% 

1.88% 

0.131 
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Table  6 

Summary  of  Cross-Validation  for  ^  +  1  ^ 

_Speannan  Rank  Correlannn 


N=250  R=0.i 
Model  I 


^  ^  S  Difference  g  + 


^  i  J  g  Difference 


N=250  R=0  1 
Model  1 


N=500  R=0,5 
Model  1 


N=500  R=0.1 
Model  1 


0.286 

226 

0  060 

0  281 

0  250 

0  031 

0  340 

0  192 

0  148 

0  34] 

0  228 

0  113 

0  361 

il  144 

0  217 

0  412 

n  178 

0  234 

0  400 

1 1 

'1  089 

0  401 

3^7 

il  044 

0467 

0  273 

0  194 

0  488 

0  308 

0  180 

0  489 

•'198 

•1  291 

0  545 

250 

0  295 

0  403 

<'358 

0  045 

0  399 

0  576 

0  022 

0.398 

0  282 

0  115 

0  429  .. 

0  343 

0  086 

0412  ' 

0  220 

0  193 

0  459 

0  274 

0  186 

0516 

0  444 

0  072 

0.532 

0  498 

0033 

0.520 

0  377 

0.143 

0  565 

0  445 

0  121 

0  528 

0  292 

0  236 

0  597 

0  .357 

0  240 

0  306 

0  245 

0  061 

0  284 

0  249 

0  034 

0.354 

0  204 

0  150 

0  355 

0218 

0  137 

0.387 

0  141 

0  246 

0  420 

0  176 

0  244 

0410 

0  318 

0.092 

0  408 

• '  356 

0  052 

0.463 

0  263 

0  200 

0.483 

0  312 

0  171 

0  496 

0  185 

0311 

0  55! 

0  245 

0  306 

0  417  0  176 
0  398  0  302 


represents  selected  sample  size. 
^  represents  selection  raao 
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N=250  R=0.5 
Model  1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

N=250  R=0.1 
Model  1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

N=500  R=0  5 
Model  1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

M=500  R=0,1 
Model  1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

Note .  N 
R 


Table  7 

Summary  of  Cross-Validation  for  g  +  2  5 


_  Spearman  Rank  Correlation _ 

- - - ^ _  _ _ SCTle2 

^  Difference  Terence  ~  g^2s 


_ _ PRESS _ 

Sample  1 _ Sanple2 

g  Difference  g  +  2  s  ^ 


Difference 


0.354 

0  352 

0.002 

0403 

0,377 

0.026 

0.311 

0  290 

0,021 

0.379 

0.315 

0.064 

0.300 

0.254 

0  045 

0.377 

0  297 

0.080 

0.477 

0.468 

0.009 

0.533 

0  492 

0040 

0.421 

0397 

0.024 

0.503 

0.409 

0,093 

0.383 

0.341 

0.042 

0.485 

0.368 

0,118 

0,235 

0.238 

-0,003 

0.294 

0  257 

0  036 

0.245 

0  179 

0.066 

0,321 

0  212 

0.110 

0  248 

0  181 

0  068 

0.325 

0.205 

0  120 

0.356 

0.333 

0.023 

0.421 

0.352 

0.070 

0.342 

0  256 

0  086 

0425 

0  268 

0  158 

0.331 

0  243 

0088 

0,427 

0.272 

0.155 

0.371 

0.355 

0.016 

0,401 

0  374 

0.028 

0.328 

0.287 

0  041 

0.394  : 

0.304 

0.089 

0.313 

0.264 

0.048 

0388 

0298 

0.091 

0.488 

0.467 

0.021 

0.543 

0.490 

0.053 

0.422 

0,370 

0.053 

0,507 

0  402 

0.105 

0,405 

0  339 

0.066 

0  495 

373 

0.122 

0.265 

0.241 

0,024 

0.303 

0  253 

0  049 

0.253 

0  195 

0.058 

0.327 

0210 

0  116 

0.267 

0  189 

0  078 

0.335 

0  204 

0.131 

0.359 

0  325 

0.034 

0.434 

0  352 

0.082 

0.348 

0.252 

0.096 

0.441 

0  285 

0.156 

0.333 

0  231 

0  103 

0.446 

0  261 

0.185 

represents  selected  sample  size, 
represents  selection  ratio. 


0.352 

0.354 

-0.002 

0.390 

0.359 

0.031 

0.314 

0.281 

0.033 

0,375 

0.308 

0.067 

0.292 

0265 

0.027 

0,370 

0.290 

0.080 

0.477 

0.461 

0.016 

0.531 

0.489 

0.042 

0.389 

0.384 

0.005 

0.497 

0.403 

0.094 

0,381 

0.331 

0.051 

0,483 

0.379 

0.104 

0.246 

0.239 

0.007 

0.304 

0.257 

0.047 

0.256 

0.194 

0.062 

0.310 

0.200 

0.110 

0  222 

0.148 

0.074 

0.311 

0.193 

0.118 

0,353 

0.329 

0.024 

0.415 

0.342 

0.073 

0.339 

0.239 

0  100 

0.421 

0.264 

0.158 

0,326 

0.230 

0.097 

0.425 

0.259 

0.166 

0.369 

0.352 

0.017 

0.402 

0370 

0.032 

0.330 

0,289 

0.041 

0.390 

0.308 

0.082 

0.318 

0.278 

0,041 

0.390 

0.300 

0.090 

0.494 

0.474 

0.020 

0.545 

0.496 

0.049 

0.415 

0  365 

0.050 

0  507 

0  399 

0.108 

0.405 

0.338 

0.067 

0.494 

0369 

0.125 

0.266 

0,243 

0.023 

0.309 

0  252 

0.058 

0  259 

0.198 

0.061 

0.320 

0.211 

0.109 

0  259 

0  177 

0.082 

0  327 

0.208 

0.118 

0.363 

0  326 

0.037 

0  421 

0.347 

0.075 

0.343 

0.243 

0.099 

0.443 

0.277 

0.166 

0.332 

0,212 

0.120 

0.445 

0.265 

0.180 

0.683 

0.683 

0.001 

0.655 

0.661 

-0.006 

0.705 

0.710 

-0.005 

0.672 

0.689 

-0.017 

0.836 

0.847 

-0.011 

0.797 

0.822 

-0.026 

0.485 

0.488 

-0.003 

0,455 

0.468 

-0.013 

0.512 

0.512 

0.000 

0.491 

0.519 

-0.027 

0.632 

0.643 

-0.011 

0.581 

0.623 

-0.042 

0.687 

0.687 

0.000 

0,663 

0.669 

-0  006 

0.702 

0.709 

-0.007 

0.676 

0.697 

-0.021 

0.833 

0.845 

-0,012 

0,789 

0.815 

-0.026 

0.482 

0.486 

-0.003 

0.453 

0.467 

-0.014 

0,515 

0.529 

-0.015 

0.482 

0.515 

-0.033 

0.623 

0.641 

-0.018 

0.577 

0.616 

-0.040 

0.669 

0  672 

-0.004 

0.651 

0.660 

-0.008 

0.693 

0.702 

-0.009 

0,667 

0.691 

-0.025 

0  828 

0.841 

-0.012 

0.787 

0.815 

-0.029 

0.479 

0.485 

-0.007 

0.450 

0.467 

-0.017 

0.525 

0.539 

-0.014 

0.482 

0.513 

-0.031 

0.623 

0.642 

-0.019 

0.571 

0.613 

-0  042 

0,681 

0,685 

-0.004 

0,662 

0.671 

-0.009 

0.699 

0.707 

-0.008 

0.666 

0.690 

-0.024 

0,827 

0.842 

-0.015 

0.789 

0,820 

-0.031 

0.477 

0483 

-0.006 

0.449 

0466 

-0.017 

0.520 

0.537 

-0.016 

0.477 

0.512 

-0.035 

0.622 

0,643 

-0.021 

0.568 

0614 

-0.046 

0.677 

0.678 

-0.001 

0.662 

0.670 

-0.007 

0.701 

0.709 

-0.008 

0.677 

0.695 

-0.018 

0.835 

0.843 

-0.008 

0.790 

0.814 

-0.024 

0.480 

0.484 

-0,003 

0.458 

0.473 

-0.014 

0.536 

0.535 

0.002 

0.488 

0.517 

-0.029 

0.634 

0.648 

-0.015 

0.581 

0.617 

-0.036 

0.679 

0,679 

0.000 

0.665 

0.675 

-0.010 

0.698 

0.708 

-0.010 

0.667 

0.686 

-0.020 

0.843 

0.853 

-0,010 

0.795 

0.820 

-0.025 

0.480 

0.483 

-0.004 

0.455 

0.470 

-0.015 

0.521 

0,536 

-0.016 

0.481 

0.513 

-0,032 

0.628 

0.648 

-0.020 

0.577 

0.615 

-0.039 

0.672 

0,677 

-0.004 

0.649 

0.659 

-0.010 

0.692 

0.701 

-0.010 

0.665 

0.688 

-0.023 

0.827 

0.839 

-0.012 

0.787 

0.816 

-0.029 

0.479 

0.485 

-0.006 

0.451 

0.469 

•0.017 

0.517 

0.530 

-0.012 

0.481 

0.512 

-0.032 

0.619 

0.638 

-0.019 

0.569 

0.610 

-0.042 

0,671 

0.675 

-0.004 

0.654 

0.666 

-0.012 

0.694 

0.703 

-0.010 

0.668 

0.689 

-0.021 

0.821 

0.836 

-0.015 

0.786 

0.816 

-0.029 

0.478 

0.485 

-0,007 

0.453 

0.467 

-0.014 

0.514 

0.530 

-0.016 

0.473 

0.509 

-0.036 

0.621 

0.645 

-0.024 

0.566 

0.612 

-0.045 
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Table  8 

Summary  of  Cross-Validation  for  g  +35 


Spearman  Rank  Correlation _  _ _ PRESS 


Sample  1 _ Sample  2 _  _ Sanple  1 


^  +  3  s 

8 

Difference 

g  +  3s- 

8 

Difference 

8  +3s 

_  T - 

8  ■ 

Difference 

N=250 

R=0.5 

Model 

1 

0.352 

0.359 

-0.007 

0.364 

0.372 

-0.008 

0.969 

0.966 

0.003 

2 

0.388 

0.379 

0.009 

0.382 

0.365 

0.017 

0.940 

0,942 

-0.002 

3 

0.342 

0.285 

0.057 

0.339 

0.294 

0.045 

1.045 

1.068 

-0.023 

4 

0.415 

0.307 

0.108 

0.399 

0.305 

0.094 

1.017 

1.062 

-0.045 

5 

0.331 

0.241 

0.091 

0.343 

0.246 

0.097 

1.058 

1.086 

-0.028 

6 

0.407 

0.275 

0.133 

0.406 

0.260 

0.145 

1.020 

1.075 

-0.056 

7 

0.474 

0.463 

0.011 

0.465 

0.461 

0.004 

0.684 

0.688 

-0.004 

8 

0.528 

0.496 

0,032 

0.519 

0,492 

0.027 

0.652 

0.666 

-0.014 

9 

0.451 

0.381 

0.071 

0.430 

0.350 

0.080 

0.772 

0.797 

-0,025 

10 

0.528 

0.380 

0.148 

0.533 

0.387 

0.146 

0.713 

0.781 

-0.067 

11 

0.470 

0.328 

0,143 

0.459 

0.332 

0.127 

0.786 

0.845 

-0.059 

12 

0.519 

0.340 

0.179 

0.523 

0.308 

0.214 

0.744 

0.828 

-0.084 

N=250 

Model 

R=0.1 

1 

0.232 

0.239 

-0.007 

0.241 

0.243 

-0.002 

0.972 

0.968 

0,005 

2 

0.282 

0.265 

0.017 

0.272 

0.241 

0.031 

0.942 

0.946 

-0.004 

3 

0.268 

0.185 

0.083 

0.263 

0.178 

0.085 

1.048 

1.067 

-0.019 

4 

0.339 

0.190 

0.149 

0.335 

0.187 

0.149 

1.010 

1.055 

-0.044 

5 

0.300 

0.170 

0.130 

0.296 

0.162 

0.134 

1.061 

1.095 

-0.035 

6 

0.352 

C.166 

0.186 

0.353 

0.168 

0.185 

1.018 

1.074 

-0.056 

7 

0.339 

0.329 

0.010 

0.333 

0.315 

0.018 

0.687 

0.689 

-0.001 

8 

0.398 

0.340 

0.058 

0.395 

0.339 

0.056 

0.652 

0.666 

-0.014 

9 

0.385 

0.262 

0.123 

0.370 

0.255 

0.115 

0.771 

0.808 

-0.036 

10 

0.486 

0.269 

0.218 

0.455 

0.251 

0.203 

0.712 

0.786 

-0.074 

11 

0.412 

0.226 

0.186 

0.393 

0.193 

0.200 

0.782 

0.838 

-0.056 

12 

0.497 

0.228 

0.268 

0.487 

0.222 

0.264 

0.743 

0.837 

-0.094 

N=500 

Model 

R=0.5 

1 

0.365 

0.364 

0.001 

0.359 

0.350 

0.009 

0.958 

0.958 

0.000 

2 

0.392 

0.370 

0.022 

0.391 

0.371 

0.020 

0.933 

0.943 

-0.010 

3 

0.350  < 

0.284 

0.067 

0.359 

0.286 

0.073 

1.034 

1.058 

-0.024 

4 

0.413 

0.295 

0.118 

0.401 

0.287 

0.114 

0.999 

1.052 

-0.053 

5 

0.359 

0.254 

0.104 

0.360 

0.242 

0.118 

1.056 

1.095 

-0.039 

6 

0.419 

0.264 

0.155 

0.423 

0.263 

0.160 

1.008 

1.075 

-0.067 

7 

0.487 

0.467 

0.020 

0.489 

0.473 

0.017 

0.680 

0.689 

-0.009 

8 

0.532 

0.493 

0.040 

0.526 

0.492 

0.034 

0.647 

0.666 

-0.019 

9 

0.465 

0.366 

0,099 

0.452 

0.355 

0.097 

0.755 

0.795 

-0.040 

10 

0.539 

0.382 

0.157 

0.548 

0.391 

0.157 

0.711 

0.781 

-0.070 

11 

0.466 

0.325 

0.141 

0.465 

0.323 

0.142 

0.785 

0.841 

-0.056 

12 

0.542 

0.329 

0.213 

0.529 

0.330 

0.200 

0.737 

0.834 

-0.097 

N=500 

Model 

R-0.1 

1 

0.264 

0.250 

0.014 

0.263 

0.243 

0.020 

0.955 

0.958 

-0.003 

2 

0.296 

0.256 

0.040 

0.303 

0.260 

0.043 

0.937 

0.946 

-0.009 

3 

0.292 

0.183 

0.109 

0.293 

0.196 

0.098 

1.027 

1.058 

-0.030 

4 

0.366 

0.201 

0.165 

0.361 

0.202 

0.159 

0.997 

1.052 

-0.054 

5 

0.315 

0.170 

0.145 

0.314 

0.166 

0.149 

1.048 

1.090 

-0.042 

6 

0.377 

0.180 

0.197 

0.381 

0.173 

0.208 

1.018 

1.083 

-0.066 

7 

0.353 

0,319 

0.035 

0.364 

0.322 

0.042 

0.676 

0.684 

-0.008 

8 

0.417 

0.349 

0.068 

0.412 

0.353 

0.059 

0.644 

0.663 

-0.020 

9 

0.395 

0.253 

0.142 

0.390 

0.251 

0.140 

0.763 

0.806 

-0.043 

10 

0.484 

0.265 

0.220 

0.485 

0,262 

0.223 

0.711 

0.786 

-0.075 

11 

0.415 

0.217 

0.198 

0.416 

0.228 

0.188 

0.778 

0.839 

-0.060 

12 

0.508 

0.235 

0.274 

0.502 

0.220 

0.282 

0.730 

0.831 

-0.101 

Note .  N  represents  selected  sample  size. 
R  represents  selection  ratio. 


g  +2s 


0.959 

0.943 

1.047 

1.013 

1.054 

1.019 

0.696 

0.656 

0.766 

0.720 

0.784 

0.747 


0.977 

0.955 

1.046 

1.002 

1.063 

1.019 

0.691 

0.647 

0.766 

0.724 

0.782 

0.741 


0.960 

0.940 

1.031 

0.997 

1.049 

1.011 

0.679 

0.642 

0.766 

0.709 

0.785 

0.741 


0.952 

0.941 

1.028 

1.000 

1.050 

1.015 

0.674 

0.643 

0.759 

0.707 

0.785 

0.728 


Sanple  2 _ 

g  Difference 


0.953 

0.006 

0.949 

-0.006 

1.065 

-0.019 

1.054 

-0.041 

1.092 

-0.038 

1.081 

-0.062 

0.697 

0.000 

0.667 

-0.011 

0.793 

-0.028 

0.791 

-0.071 

0.837 

-0.052 

0.842 

-0.095 

0.974 

0.004 

0.961 

-0.006 

1.064 

-0.018 

1.046 

-0.044 

1.097 

-0.033 

1.074 

-0.056 

0.695 

-0.003 

0.662 

-0.015 

0.801 

-0.035 

0.787 

-0.063 

0.834 

-0.052 

0,833 

-0.092 

0.964 

-0.004 

0.948 

-0.009 

1.060 

-0.029 

1.047 

-0.050 

1.091 

-0.042 

1.082 

-0.071 

0.686 

-0.008 

0.659 

-0.016 

0.804 

-0.038 

0.786 

-0.076 

0.840 

-0.055 

0.832 

-0,090 

0.956 

-0.004 

0.953 

-0.012 

1.057 

-0.029 

1.054 

-0.054 

1.093 

-0.043 

1.086 

-0.071 

0.684 

-0.010 

0.660 

-0.017 

0.800 

-0.041 

0.783 

-0.077 

0.843 

-0.058 

0.827 

-0.099 
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Table  9 

The  Number  of  Incumbents  in  the  Army  Project  A  and 
the  Marine  Corps  JPM  Studies 


Enlisted  Job 

N 

Army  Project  A 

I.  Infantryman  (I  IB) 

491 

2.  Cannon  Crewman  (I3B) 

464 

3.  Tank  Crewman  (19E) 

394 

4.  Radio  Operator  (3 1C) 

289 

5.  Vehicle  Mechanic  (63B) 

478 

6.  Motor  Transport  (64C) 

507 

7.  Administrative  (71 L) 

427 

8.  Medical  (91  A) 

392 

9.  Military  Police  (95B) 

597 

Total 

4,039 

Marine  Corps  JPM 

10.  Helicopter  Mechanic 

439 

1 1 .  Automotive  Mechanic 

694 

Total 

1,133 
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Table  10 

Summary  of  Tests  in  the  Studies 


Subtest 

Description 

Number  of  Items 

ASVAB  subtests 

GS 

General  Science 

25 

AR 

Arithmetic  Reasoning 

30 

WK 

Word  Knowledge 

35 

PC 

Paragraph  Comprehension 

15  ' 

NO 

Numerical  Operations 

50 

CS 

Coding  Speed 

84 

AS 

Auto  &  Shop  Information 

25 

MK 

Mathematical  Knowledge 

25 

MC 

Mechanical  Comprehension 

25 

El 

Electronics  Information 

20 

Armv  Proiect  A  subtests 

ASSEM.OBJ 

Assembling  Objects 

32 

REASON 

Figural  Reasoning 

35 

MAZE 

Maze  Test 

24 

OBJ.ROT 

Object  Rotation  Test 

90 

ORIENT/ 

Orientation  Test 

24 

MAP 

Map  Test 

20 

TARGET  1 

Target  Tracking  Test  1 

18 

TARGET2 

Target  Tracking  Test  2 

18 

IDENT.D 

Target  Identification,  Time 

36 

IDENT.H 

Target  Identification,  Hits 

30 

MEM.DIS 

Memory  Search  Test,  Time 

MEM.HIT 

Memory  Search  Test,  Hits 

ECAT  subtests 

ID 

Integrating  Details 

40 

SM 

Sequential  Memory 

35 

AO 

Assembling  Objects 

32 

FR 

Figural  Reasoning 

35 

SO 

Spatial  Orientation 

24 

T1 

One-Hand  Tracking 

18 

T2 

Two-Hand  Tracking 

18 

TI 

Target  Identification 

36 
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Table  1 1 

Factor  Scores  Description 


1 .  Infantryman  (  n  =  49 1) 


Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

48.00 

4.92 

34.89 

60.06 

0.927 

Gc 

14.32 

4.54 

1.15 

29.17 

0.900 

Gv/Mech 

14.86 

4.30 

0.15 

26.94 

0.865 

Speed 

15.40 

3.51 

4.39 

24.85 

0.840 

Math 

-5.43 

3.64 

-18.15 

6.69 

0.776 

Psymotor 

11.60 

1.09 

8.89 

15.06 

0.634 

G 

54.10 

5.42 

40.19 

66.32 

Hands-On 

54.15 

5.68 

36.50 

69.98 

2.  Cannon  Crewman  (  n  =  464) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

46.19 

4.50 

32.99 

58.33' 

0.927 

Gc 

12.80 

5.16 

-8.74 

27.93' 

0.900 

Gv/Mech 

12.99 

4.81 

-1.71 

26.37 

0.865 

Speed 

15.31 

3.62 

2.47 

26.63 

0.840 

Math 

-5.24 

3.44 

-13.71 

7.93 

0.776 

Psymotor 

11.80 

1.22 

8.74 

15.98 

0.634 

G 

50.90 

5.68 

37.04 

67.84 

Hands-On 

48.01 

8.72 

20.62 

73.86 

3. 

Tank  Crewman  ( n  =  394) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

47.65 

4.72 

34.74 

58.39 

0.927 

Gc 

14.25 

4.93 

-2.76 

25.56 

0.900 

Gv/Mech 

15.34 

4.27 

0.41 

25.95 

0.865 

Speed 

14.88 

3.64 

5.18 

24.37 

0.840 

Math 

-5.17 

3.44 

-15.74 

5.65 

0.776 

Psymotor 

11.62 

1.08 

9.05 

14.90 

0.634 

G 

53.84 

5.60 

41.00 

67.53 

Hands-On 

59.87 

5.65 

39.88 

72.10 
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Table  1 1 

Factor  Scores  Description  (continued) 
4.  Radio  Operator  ( n  =  289) 


Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

48.25 

4.52 

37.50 

59.40 

0.927 

Gc 

13.80 

4.53 

0.85 

26.33 

0.900 

Gv/Mech 

13.42 

5.14 

-3.14 

26.75 

0.865 

Speed 

17.33 

3.34 

7.65 

24.80 

0.840 

Math 

-5.67 

3.58 

-15.67 

4.23 

0.776 

Psymotor 

11.76 

1.07 

9.00 

14.87 

0.634 

G 

53.62 

5.15 

41.49 

65.38 

Hands-On 

54.19 

6.03 

33.60 

68.37 

5.  Vehicle  Mechanic  ( 

n  =  478) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

46.76 

4.54 

35.51 

57.97 

0.927 

Gc 

13.34 

4.36 

-0.27 

24.72 

0.900 

Gv/Mech 

16.60 

4.65 

2.45 

27.80 

0.865 

Speed 

15.46 

3.51 

5.87 

25.93 

0.840 

Math 

-5.53 

3.49 

-17.53 

4.27 

0.776 

Psymotor' 

11.93 

1.13 

8.64 

15.83 

0.634 

G 

52.96 

5.25 

38.81 

65.62 

Hands-On 

65.14 

3.70 

46.20 

72.75 

6.  Motor  Transport  (  n  =  507) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

45.51 

4.66 

33.03 

58.21 

0.927 

Gc 

12.68 

4.67 

-1.68 

24.57 

0.900 

Gv/Mech 

15.18 

4.96 

-3.05 

26.72 

0.865 

Speed 

15.42 

3.64 

5.38 

25.94 

0.840 

Math 

-5.84 

3.30 

-16.58 

4.32 

0.776 

Psymotor 

11.85 

1.18 

8.78 

15.58 

0.634 

G 

50.86 

5.06 

39.39 

65.14 

Hands-On 

55.10 

5.81 

33.07 

68.65 
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Table  1 1 

Factor  Scores  Description  (continued) 
7.  Administrative  (  n  =  427) 


Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

47.64 

4.67 

33.92 

59.85 

0.927 

Gc 

13.20 

4.18 

-2.38 

23.84 

0.900 

Gv/Mech 

9.14 

4.93 

-4.98  : 

25.06 

0.865 

Speed 

18.09 

3.08 

7.67 

25.21 

0.840 

Math 

-5.66 

3.72 

-17.98  ; 

6.39 

0.776 

Psymotor 

12.27 

1.16 

8.95 

15.91 

0.634 

G 

51.04 

5.32 

38.22 

65.38 

Hands-On 

45.63 

6.32 

25.55 

63.64 

8.  Medical  ( 

n  =  392) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

49.15 

4.17 

35.99 

59.49 

0.927 

Gc 

15.25 

4.09 

-1.38  ■ 

28.62 

0.900 

Gv/Mech 

11.98 

5.14 

-1.09 

24.42 

0.865 

Speed 

15.14 

3.92 

3.80 

26.49 

0.840 

Math 

-5.44 

3.52 

-14.63 

4.69 

0.776 

Psymotor 

11.95 

1.17 

8.97  : 

15.22 

0.634 

G 

54.28 

4.40 

44.55 

65.34 

Hands-On 

55.77 

5.38 

37.49 

69.56 

9.  Military  Police  ( n  =  597) 

Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

49.32 

3.68 

37.81 

58.53 

0.927 

Gc 

15.60 

3.71 

2.98 

26.44 

0.900 

Gv/Mech 

14.95 

4.42 

-0.09 

27.07 

0.865 

Speed 

15.17 

3.46 

3.37 

25.08 

0.840 

Math 

-5.03 

3.24 

-14.41 

6.62 

0.776 

Psymotor 

11.55 

1.02 

8.70 

15.19 

0.634 

G 

56.02 

3.67 

46.38 

65.27 

Hands-On 

54.12 

4.71 

37.99 

64.87 
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Table  1 1 

Factor  Scores  Description  (continued) 
10.  Helicopter  Mechanic  (  n  =  439) 


Variable 

Mean 

Std  Dev 

Minimum 

Maximum 

Determinacy 

Gf 

27.81 

4.44 

14.35 

37.28 

0.946 

Gc 

25.83 

4.04 

7.81 

34.73 

0.902 

Gv/Mech 

24.67 

4.14 

:  13.67 

34.36 

0.861  ; 

Speed 

30.49 

3.70 

8.01 

41.93 

0.859 

Math 

5.42 

2.94 

;  -4.15 

13.27 

0.742  “ 

Psymotor 

-57.71 

6.54 

-79.35 

-42.60 

0.916 

G 

50.78 

3.90 

40.22 

61.45 

Hands-On 

76.92 

7.98 

•'  45.00 

94.00 

1 1 .  Automotive  Mechanic  (  n  =  694) 


Variable 

Mean 

Std  Dev 

..  Minimum 

Maximum 

Determinacy 

Gf 

29.32 

4.40 

••  18.51 

41.57 

0.935  • 

Gc 

21.67 

4.21 

•  -0.08 

33.49 

0.887 

Gv/Mech 

20.42 

4.03 

6.12 

30.77 

0.858 

Speed 

32.65 

3.70 

20.50 

42.17 

0.853 

Math 

4.74 

3.09 

-5.14 

13.99 

0.747 

Psymotor 

-58.84 

6.88 

-83.29 

-41.49 

0.883 

G 

48.58 

4.22 

37.62 

61.79 

Hands-On 

77.82 

7.61 

43.00 

93.00 
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Table  12 

Estimates  for  the  Standardized  Prediction  Equation  and  Adjusted  R  Squares 


Job 

N 

Gf 

Gc 

Gv/Mech 

Speed 

Math 

Psymotor 

Adjusted 

Project  A 

1.  Infantryman 

491 

0.258® 

0.081 

0.218 

0.023 

0.079 

-0.075 

0.178 

5.922^ 

1.854 

5.087 

0.512 

1.764 

-1.679 

2.  Cannon  Crewman 

464 

0.140  i 

0.034 

0.113 

0.001 

0.021. 

-0.027 

o;o5o 

2.780 

0.720 

2.367 

0.020 

0.433 

-0.566 

3.  Tank  Crewman 

394 

0.242  ' 

0.021 

0.223 

-0.075 

0.080 

-0.028 

0.190 

4.853 

0.431 

4.645 

-1.488 

1.596 

-0.562 

4.  Radio  Operator 

289 

0.338  V 

0.213 

0.205 

0.158 

0.090' 

0.057 

0.‘‘229 

5.034 

3.271 

3.339 

2.112 

1.518 

0.983 

5.  Vehicle  Mechanic 

478 

0.117 

0.057 

0.236 

-0.093 

0.046 

-0.110 

0.161 

2.581 

1.256 

5.264 

-1.983 

0.990 

-2.304 

6.  Motor  Transport 

507 

0.318  ... 

0.069 

0.283 

-0.008 

0.002 

-0:024 

0.247 

7.524  .. 

1.533 

6.262 

-0.181 

0.051 

-0.562 

7.  Administrative 

427 

0.469 

-0.026 

0.055 

0.054 

0.041 

0.053 

0.348 

9.677 

-0.577 

1.192 

1.097 

0.905 

1.174 

8.  Medical 

392 

0.362 

0.131 

0.223 

0.086 

0.045 

0.004 

0.270 

7.033 

2.524 

4.666 

1.724 

0.880 

0.084 

9.  Military  Police 

597 

0.323 

0.063 

0.258 

-0.014 

0.130 

0.000 

0.290 

7.779 

1.470 

6.716 

-0.357 

3.327 

0.003 

Marine  Corps 

10.  Helicopter  Mechanic 

439 

0.404  , 

-0.011 

0.295 

0.036 

0.030 

-0.024 

0.519 

7.983 

-0.226 

6.056 

0.809 

0.659 

-0.530 

1 1 ,  Automotive  Mechanic 

694 

0.355 

0.135 

0.343 

0.006 

-0.005 

-0.024 

0.569 

9.667 

3.763 

10.138 

0.171 

-0.136 

-0.711 

'Standardized  regression  coefficient, 
ratio. 


Table  13 

Estimates  for  the  Standardized  Prediction  Equation  (Top  50  %) 


Job 

N 

Gf 

Gc 

Gv/Mech 

Speed 

Math 

Psymotor 

Project  A 

I.  Infantryman 

246 

0.228" 

0.123 

0.177 

0.026 

-0.028 

0.040 

3.252'’ 

1.675 

2.507 

0.403 

-0.407 

0.571 

2.  Cannon  Crewman 

232 

0.004 

-0.088 

0.061 

-0.052 

0.028 

-0.047 

0.064 

-1.246 

0.875- 

-0.737 

0.394 

-0.662 

3.  Tank  Crewman 

197 

0.036 

0.085 

0.248 

-0.099 

0.120 

-0.088 

e.- 

0.450 

1.163 

3.268. 

-1.354 

1.591 

-1.186 

4.  Radio  Operator 

145 

0.200 

0.220 

0.172 

0.085 

0.121 

0.089 

1.950 

2.267 

1.870 

0.905 

1.387 

1.003 

5.  Vehicle  Mechanic 

239 

0.086 

-0.034 

0.116 

-0.162 

0.065 

-0.088 

1.246 

-0.486 

1.667 

.2.359 

0.933 

-1.223 

6.  Motor  Transport 

254 

0.261 

0.144 

0.324 

0.006 

-0.055 

-0.051 

3.982 

2.082 

4.620 

0.087 

-0.869 

-0.781 

7.  Administrative 

214 

0.452 

0.029 

0.144 

0.048 

0.023 

0.041 

6.537 

0.422 

2.199 

0.734 

0.351 

0.635 

8.  Medical 

196 

0.288 

0.064 

0.171 

0.106 

0.124 

-0.007 

3.280 

0.822 

2.077 

1.471 

1.595 

-0.084 

9.  Military  Police 

299 

0.203 

0.011 

0.142 

-0.074 

0.241 

-0.026 

3.007 

0.175 

2.278 

-1.290 

3.983 

-0.437 

Marine  Corps 

10.  Helicopter  Mechanic 

220 

0.331 

0.034 

0.315 

0.012 

0.017 

0.062 

4.397 

0.456 

4.457 

0.185 

0.256 

0.920 

11.  Automotive  Mechanic 

347 

0.268 

0.156 

0.354 

-0.003 

0.014 

-0.002 

4.557 

2.763 

6.885 

-0.062 

0.275 

-0.035 

“Standardized  regression  coefficient, 
ratio. 
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Table  14 

Spearman  Rank-Order  Correlations 
(Predicted  Y  with  Hands-On  Job  Performance) 


Job 

N 

#  +  -55 

g  +  s 

Si 

Project  A 

1 .  Infantryman 

491 

0.351‘ 

8.277*’ 

0.338 

7.946 

0.331 

7.754 

2.  Caimon  Crewman 

464 

0.222 

4.901 

0.216 

4.743 

0.218 

4.803 

ti 

3.  Tank  Crewman 

394 

0.375 

8.014 

0.355 

7.522 

0.337 

7.085 

4.  Radio  Operator 

289 

0.402 

7.430- 

0.390 

7.177 

0.391 

7.206 

5.  Vehicle  Mechanic 

478 

0.322 

7.410 

0.308 

7.059 

0.267 

6.043 

6.  Motor  Transport 

507 

0.414 

10.220 

0,409 

10.070 

0.365 

8.822 

7.  Administrative 

427 

0.439 

10.079 

0.430 

9.830 

0.374 

8.320 

8.  Medical 

392 

0.377 

8.044 

0.368 

7.815 

0.369 

7.842 

9.  Military  Police 

597 

0.394 

10.464 

0.391 

10.364 

0.367 

9.613 

Marine  Corps 

10.  Helicopter  Mechanic 

439 

0.400 

9.113 

0.400 

9.111 

0.293 

6.415 

1 1 .  Automotive  Mechanic 

694 

0.488 

0.487 

0.395 

14.724 

14.651 

11.327 

‘Spearman  rank-order  correlation, 
ratio. 
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Table  1 5 

Job  Classifications 


Job  . 

Selection 

Failure 

Failure 

Success 

Success 

“ 

Proportion  of 

(Top  50%) 

N 

Method 

Reject 

Accept 

Reject 

Accept 

Sensitivity 

Specificity 

Correct  Decisions 

Project  A 

i.  Infantryman 

491 

g  +  5s 

148 

97 

97 

149 

60.57% 

60.41% 

60.49% 

149 

96 

96 

150 

60.98% 

60.82% 

60.90% 

Si 

149 

96 

96 

150 

60.98% 

60.82% 

60.90% 

2.  Cannon  Crewman 

464 

g  + 5s 

125 

107 

107 

125 

53,88% 

53.88% 

53.88% 

g 

141 

91' 

91 

141 

60.78% 

60.78% 

60.78% 

SI 

140 

92 

92 

140 

60.34% 

60.34% 

60.34% 

3,  Tank  Crewman 

394 

g  +  5s 

123 

74 

74 

123 

62.44% 

62.44% 

62.44% 

li 

120 

77. 

77 

,  120 

60.91% 

60.91% 

60.91% 

si 

125 

72 

72 

125 

63.45% 

63,45% 

63.45% 

4:  Radio  Operator 

289 

95 

49 

49 

96 

66.21% 

65.97% 

66.09% 

g-¥S 

88 

56 

56 

89 

61.38% 

61.11% 

61.25% 

Si 

96 

48 

48 

97 

66.90% 

66.67% 

66.78% 

5.  Vehicle  Mechanic 

478 

+ 

155 

84, 

84 

•  155 

64.85% 

64.85% 

64.85% 

i  +  i 

144 

95. 

95 

.  144 

60.25% 

60.25% 

60.25% 

Si 

148 

91 

91 

148 

61.92% 

61.92% 

61.92% 

6.  Motor  Transport 

507 

g  +  5s 

160 

93 

93 

161 

63.39% 

63.24% 

63.31% 

g  +  i 

154 

99 

99 

155 

61.02% 

60.87% 

60.95% 

' 

SI 

158 

95 

95 

159 

62.60% 

62.45% 

62.52% 

7.  Administrative 

427 

g  + 5s 

136 

77 

77 

137 

64.02% 

63.85% 

63.93% 

g  +  s 

135 

78 

78 

136 

63.55% 

63.38% 

63.47% 

Si 

128 

85 

85 

129 

60.28% 

60.09% 

60.19% 

8.  Medical 

392 

g  +  5s 

123 

73 

73 

123 

62.76% 

62.76% 

62.76% 

prr 

121 

75 

75 

121 

61.73% 

61.73% 

61.73% 

Si 

122 

74 

74 

122 

62.24% 

62.24% 

62.24% 

9.  Military  Police 

597 

g  +  5s 

191 

107 

107 

192 

64.21% 

64.09% 

64.15% 

g  +  S 

186 

112 

112 

187 

62.54% 

62.42% 

62.48% 

SI 

186 

112 

112 

187 

62.54% 

62.42% 

62.48% 

Marine  Corps 

10.  Helicopter  Mechanic 

439 

g  +  5s 

140 

79 

79 

141 

64.09% 

63.93% 

64,01% 

g  +  s 

140 

79 

79 

141 

64.09% 

63.93% 

64.01% 

SI 

131 

88 

88 

132 

60.00% 

59.82% 

59.91% 

1 1.  Automotive  Mechani 

694 

g  + 5s 

240 

107 

107 

240 

69.16% 

69.16% 

69.16% 

g  +  s‘ 

240 

107 

107 

240 

69.16% 

69.16% 

69.16% 

SI 

225 

122 

122 

225 

64.84% 

64.84% 

64.84% 

Note .  Regression  based  on  selected  sample  from  single-factor  model. 
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Figure  2 

General  Form  of  the  Decision  Table 


;  Failure 

Future  Performance 

Success 

Reject  Accept 
Selection  Decision 

Sensitivity  =  TP  /  (FN  +  TP) 

Sjjecificity  =  TN  /  (TN  +  FP) 

Proportion  of  Correct  Decisions 
=  (TP  +  TN)/N 

=  (TP  +  TN)  /  (FN  +  TP  +  TN  +  FP) 
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Figures 

Data  Pattern  for  Cross-Validation  in  the  Selected  Sample 


Si  s  ^  y  yg+s  Pgj 


Estimates  of  Regression  Slope 
>'  =  d  +  p^xg  +  p^xf 

y  =  a  +  ^gjxgj 

Estimates  of  Criterion 
yg+s  +  xg  +  p^  X 

hi  =«  +  Pg/  xi/ 

PRESS  = 


Note:  represents  the  sample  size  for  the  selected  subjects 

Ni=Ns/2. 
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Figure  5 

Hierarchical  Factor  Model 
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Figure  6 

Data  Pattern  for  the  Artificial  Data 


X3  ...  X9  xio  y  g  s  gj  g  s 


Non-selected  Sample 
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Plots  of  Classification  for  +  1 


Proportion  of  Correct  Decisions 
(N=250,  R=0.5) 


Proportion  of  Correct  Decisions 
(N=250,  R=0.1) 


Proportion  of  Correct  Decisions 


Proportion  of  Correct  Decisions 
(N=500,  R=0.1) 
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Figure  8 

Plots  of  Classification  for  g  +  2  s 
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rigure  9 
Plots  of  Classification  for  g 


Sensitivity 
(N=250,  R=0.5) 


Sensitivity 

(N=250,R=0.1) 


Sensitivity 
(N=500,  R=0.5) 
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Figure  9 

Plots  of  Classification  for  g  +  3  ^ 
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Plots  of  Difference  for  g  2  s  Classification 


Figure  14 

Plots  of  Spearman  Rank-Order  Correlation  for^  - 


r 


Spearman  Rank-Order  Correiation 
(N=250,R=0.1) 

08  ^ 


Spearman  Rank-Order  Correlation 
0^=500,  R=0.1) 


0.3 


Figure  16 

Plots  of  Difference  for  Cross-Validation  of  g  -t- 1  s 


Difference  for  Rank-Order  .Correlation 
(N=250,  R=0.5) 


Modd 


Difference  for  PRESS 
(N=250,  R=0.5) 


Model 


- □ - lample  1 

—  -A  —  •tample2 


Difference  for  Rank-Order  Correlation 
(N=250,R=0.1) 


Modd 


— O - nmpic  1 

-  -A-  lanpleZ 


Difference  for  Rank-Order  Correlation 
(N=500,  R=0.5) 


Modd 


— C3 - nsnplc  1 

— A-  -ramplcZ 


Difference  for  PRESS 
(N=250,  R=0.1) 


Modd 


Difference  for  PRESS 
(N=500,  R-0.5) 


— O - iwnplel 

■  -A-  iimple2 


1  7  4  3  10  9  6  5  12  11 


2  8  1  7  4  3  10  9  5  6  11  12 


Plots  of  Difference  for  Cross-Validation  of  g  +  2  - 


Difference  for  Rank-Order  Correlation 
(N=250,R-0.5) 


~0 - iKtipiel 

•tampk2 


'11  5  A  6  10-  12 


Difference  for  PRESS 
(N=250,  R=0.f  ) 


— □ - itmple  1 

—  -A—  •famplc2 


9  7  3  2  11  3 


Difference  for  Rank-Order  Correlation 
(N=250.  R=0.1) 


- O - nmple  1 

—  -A—  •  ample  2 


9  11  4  6  12  10 


Difference  for  PRESS 
(N=250,R=0.1) 


- C3 - nitfile  1 

—  -A-  •t«mpk2 


9  11  4  6  10  12 


Difference  for  Rank-Order  Correlation 
(N=500,R=0.5) 


— □ - nniplel 

-  -A-  •Mtnple2 


2  3  3  8  9  11  4  6  10  12 


Difference  for  PRESS 
(N=500,  R=0.5) 


— a - nmpitl 

—  -A—  •Mnip1e2 


7  2  3  5  9  8  11  4  6  10  12 


Difference  for  Rank-Order  Correlation 
(N=500,R=0.1) 


— O - ncnple  1 

-  -A-  nmple 2 


Difference  for  PRESS 
(N=500,R=0.1) 


— O — nmple  1 
-  -A-  -nmple 2 


11  4  0  10  12 


Figure  18 

Plots  of  Difference  for  Cross-Validation  of  g  +  3  5 


1  72839546  11  10  12 
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Appendix  A 

The  Factor  Structure 
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Table  A1 


The  Factor  Structure  for  ASVAB  and  Project  A  Subtests 

Subtest 

Gf 

Gc 

Gv/Mech 

Speed 

Math 

Psymotor 

General  Science 

0.56 

0.55 

0.24 

_ 

0.17 

Arithmetic  Reasoning 

0.67 

0.27 

0.20 

0.18 

0.31 

Word  Knowledge 

0.57 

0.72 

0.12 

Paragraph  Comprehension 

0.56 

0.54 

— 

0.22 

Numerical  Operations 

0.42 

— 

0.72 

0.16 

Coding  Speed 

0.44 

0.63 

Auto  &  Shop  Information 

0.20 

0,45 

0.70 

Mathematical  Knowledge 

0.68 

0.13 

0.16 

0.57 

0  08 

Mechanical  Comprehension 

0.57 

0.32 

0.52 

0  13 

-0  05 

Electronics  Information 

0.31 

0.51 

0.53 

0  15 

Assembling  Objects 

0.68 

0.27 

Figural  Reasoning 

0.77 

_ 

0.17 

Maze  Test 

0.56 

— 

0.29 

„ 

-0.35 

Object  Rotation  Test 

0.48 

— 

0.31 

-0.22 

Orientation  Test 

0.63 

_ 

0.28 

Map  Test  ■ 

0.72 

0.13 

0.29 

0.11 

Target  Tracking  Test  1 

-0.38 

— 

-0.35 

0.30 

Target  Tracking  Test  2 

-0.40 

— 

-0.36 

0  30 

Target  Identification,  Time 

-0.40 

_ 

-0.27 

\j  .j\j 

0  41 

Target  Identification,  Hits 

0.23 

\/.*T  1 

Memory  Search  Test,  Time 

-0.17 

0.10 

-0.06 

Memory  Search  Test,  Hits 

0.37 

— 

- 

“ 

- 

Mote .  From  'The  nature  of  the  general  factor  in  hierarchical  models  of  the  structure  of  cognitive  abilities:  alternative 
models  tested  on  data  from  regular  and  experimental  military  enlistment  tests."  by  J.  E.  Gustafsson  and  B.  Muthen,  1994, 
UCLA  Technical  Report  p.  15.  Adapted  with  permission  of  the  author. 
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Table  A2 

The  Factor  Structure  for  ASVAB  and  ECAT  Subtests 


Subtest 

Gf 

Gc 

Gv/Mech 

Speed 

Math 

Psymotor 

General  Science 

0.59 

0.52 

0.25 

__ 

0.17 

Arithmetic  Reasoning 

0.73 

0.21 

0.17 

0.17 

0.27 

Word  Knowledge 

0.58 

0.71 

0.12 

Paragraph  Comprehension 

0.54 

0.55 

_ 

0.24 

Numerical  Operations 

0.41 

— 

_ 

0.71 

0.16 

Coding  Speed 

0.41 

— 

0.66 

Auto  &  Shop  Information 

0.28 

0.38 

0.70 

Mathematical  Knowledge 

0.68 

0.12 

0.49 

0.18 

0.57 

Mechanical  Comprehension 

0.60 

0.28 

0.55 

0.14 

Electronics  Information 

0.38 

0.44 

„ 

0.14 

Assembling  Objects 

0.72 

0.29 

Integrating  Details 

0.76 

_ 

0.26 

Sequential  Memory 

0.70 

_ 

Figural  Reasoning 

0.76 

.. 

0.11 

Spatial  Orientation 

0.68 

0.26 

One-Hand  Tracking 

-0.44 

— 

-0.21 

„ 

0.68 

Two-Hand  Tracking 

-0.47 

_ 

-0.27 

0.72 

Target  Identification 

-0.38 

- 

-0.16 

~ 

~ 

0.23 

Note .  From  '  The  nature  of  the  general  factor  in  hierarchical  models  of  the  structure  of  cognitive  abilities:  alternative 
models  tested  on  data  from  regular  and  experimental  military  enlistment  tests,"  by  J.  E.  Gustafsson  and  B.  Muthen,  1994, 
UCLA  Technical  Report,  p.  12,  Adapted  with  permission  of  the  author. 
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Appendix  B 


Job  Profile  Description 
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Job  1  (Infantryman) 


TABLE  OF  TOP  BY  NEW 


TOP 


NEW 


Frequency  t 
Percent  | 
Row  Pet  [ 
Col  Pet  I 


II 


1  62  I 

I  12.63  1 

1  15.86  1 

I  62.00  1 


21 

- +- 

71  I 
14.46  I 
18.16  1 
71.00  I 


31 


4  i 


85 

17.31 

21.74 

85.00 


I  86 

[  17,52 

I  21.99 
I  86.00 


5 i  Total 


87 

17.72 

22.25 

95.60 


- +- 

- 

- +- 

11  38  I 

29  1 

15  1 

14  1 

4 

1  7.74  1 

5.91  1 

3.05  1 

2.85  1 

0.81 

1  38.00  1 

29.00  1 

15.00  1 

14.00  I 

4.00 

1  38.00  1 

29,00  1 

15.00  I 

14.00  1 

4.40 

Total  100 

100 

100 

100 

91 

20.37 

20.37 

20.37 

20.37 

18.53 

391 

79.63 


100 

20.37 


491 

100.00 


TABLE  OF  TOP  BY  SIG 


TOP 


SIG 


Frequency  I 
Percent  | 
Row  Pet  t 
Col  Pet  1 


II 


21 

— +- 


I  64  I 

I  13.03  I 

I  16.37  I 

}  64.00  1 


71 

14.46 

18.16 

71.00 


31 

- +- 

84  I 
17.11  I 
21.48  I 
84.00  I 


87 

17.72 

22.25 

87.00 


4i 

-+- 

I 

I 


Total 


I  36  1 

I  ;  7.33  I 

I  36.00  I 

I  36.00  I 


29 

5.91 

29.00 

29.00 


(  16 
I  3.26 
I  16.00 
I  16.00 


13 

2.65 

13.00 

13.00 


100 

20.37 


100  100 
20.37  20.37 


100 

20.37 


51 
— + 


85 

17.31 

21.74 

93.41 


6 

1.22 

6.00 

6.59 


91 

18.53 


Total 

391 

79.63 


100 

20.37 


491 

100.00 


TABLE  OF  TOP  BY  GEN 


TOP 


GEN 


Frequency 
Percent 
Row  Pet 
Col  Pet 


II 


21 

— +- 


31 

— +- 


41 


62  1 
12.63  1 
15.86  I 
62.00  I 


70  I 
14.26  I 
17.90  I 
70.00  I 


87 

17.72 

22.25 

87.00 


88 

17.92 

22.51 

88.00 


38  I 
7.74  I 
38.00  I 
38.00  I 


Total 


100 

20.37 


100 

20.37 


100 

20,37 


100 

20.37 


51 
— + 


84 

17.11 

21.48 

92.31 


30  I  13  I  12  I  7 

6.11  I  2.65  I  2.44  |  1.43 

30.00  I  13.00  I  12.00  I  7.00 

30.00  I  13.00  I  12.00  I  7.69 


91 

18.53 


Total 

391 

79.63 


100 

20.37 


491 

100.00 
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Job  2  (Cannon  Crewman) 


TABLE  OF  TOP  BY  NEW 


TOP 


NEW 


Frequency | 

Percent 
Row  Pet 
Col  Pet 

- ■+ - + - + - + - + - 


64  1  79 

13.79  I  17.03 
17.58  1  21.70 

64.00  I  79.00 

- + - + - 

36  I  21 

7.76  I  4.53 
36.00  I  21,00 
36.00  I  21.00 
- + - +_ 


Total 


100 

21.55 


100 

21.55 


3i 
+ 

78  I 
16.81  I 
21.43  I 
78.00  I 

- + 

22  1 
4.74  I 
22.00  I 
22.00  I 
+ 

100 

21.55 


86 

18.53 

23.63 

86.00 

14 

3.02 

14.00 

14.00 


-+ - 


- 1 - 1 - H - 


100 

21.55 


51 
-+ 
57  I 
12.28  i 
15.66  1 
89.06  1 
+ 
7  I 
1.51  I 
7.00  I 
10.94  I 
+ 
64 
13.79 


Total 

364 

78.45 


100 

21.55 


464 

100.00 


TABLE  OF  TOP  BY  SIG 


TOP 


SIG 


Frequency 
Percent 
Row  Pet 
Col  Pet 


Total 


II 

-~+- 


21 

— +- 


65 

14.01 

17.86 

65.00 


41 

— +- 


35 
,  7.54 
35.00 
35.00 


100 

21.55 


Total 


78  1 
16.81  1 
21.43  1 
78.00  1 

75 

16.16 

20.60 

75.00 

1 

I 

1 

1 

88  ( 
18.97  ! 
24.18  1 
88.00  1 

58  1 
12.50  1 
15.93  1 
90.63  1 

364 

78.45 

22  1 

25 

1 

12  1 

6  1 

100 

4.74  1 

5.39 

1 

2.59  1 

1.29  1 

21,55 

22.00  1 

25.00 

I 

12.00  1 

6.00  1 

22.00  1 

25.00 

1 

12.00  1 

9.38  1 

100 

100 

100 

64 

464 

21.55 

21.55 

21.55 

13.79 

100.00 

TABLE  OF  TOP  BY  GEN 


TOP 


GEN 


Frequency 
Percent 
Row  Pet 
Col  Pet 


Total 


1| 

21 

31 

4! 

51 

Total 

68 

I 

71  ! 

83 

1 

87  1 

55 

“+ 

1 

364 

14.66 

1 

15.30  1 

17.89 

1 

18.75  I 

11.85 

1 

78.45 

18.68 

1 

19.51  1 

22.80 

1 

23.90  1 

15.11 

1 

68.00 

1 

71.00  1 

83.00 

1 

87.00  1 

85.94 

1 

32 

1 

29  1 

17 

I 

13  1 

9 

-+ 

1 

100 

6.90 

1 

6.25  1 

3.66 

1 

2.80  1 

1.94 

1 

21.55 

32.00 

! 

29.00  1 

17.00 

1 

13.00  1 

9.00 

1 

32.00 

1 

29.00  1 

17.00 

1 

13.00  1 

14.06 

1 

100 

21.55 

100 

21.55 

100 

21.55 

100 

21.55 

64 

13.79 

464 

100.00 
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Job  3  (Tcink  Crevraian) 


TABLE  OF  TOP  BY  NEW 


TOP  NEW 


Frequency 

Percent 


Row  Pet 
Col  Pet 


+• 


1 


Total 


11 


61  i 
15.48  I 
20.75  I 
61,00  I 


39  I 
9.90  i 
39.00  I 
39.00  I 


100 

25.38 


21 


68  I 
17.26  I 
23.13  I 
68.00  I 


32  I 
8.12  1 
32.00  I 
32.00  I 


100 

25.38 


31 


78  I 
19.80  I 
26.53  I 
78.00  i 


22  I 
5.58  I 
22.00  I 
22.00  I 


100 

25.38 


41 


87  1 
22.08  I 
29.59  I 
92.55  I 


7  I 
1.78  I 
7.00  I 
7.45  I 


94 

23.86 


Total 

294 

74.62 


100 

25.38 


394 

100.00 


TABLE  OF  TOP  BY  SIG 


TOP  SIG 


Frequency | 
Percent  | 
Row  Pet  1 
Col  Pet  1 

11 

0 

1 

59 

i 

i 

14.97 

1 

1 

20.07 

! 

1 

59.00 

1 

1  ' 

1 

41 

1 

1 

10.41 

I 

1 

41.00 

1 

1 

+— 

41.00 

1 

Total 

100 

25.38 


21  31 

- + - 4.. 

71  }  78  I 

18.02  I  19.80  I 
24.15  I  26.53  | 
71.00  I  78.00  I 


29  I  22  1 

7.36  I  5.58  I 
29.00  I  22.00  I 
29.00  I  22.00  I 


100  100 
25.38  25.38 


4| 


86  I 
21.83  I 
29.25  1 
91.49  1 


8  I 
2.03  I 
8.00  I 
8.51  1 


94 

23.86 


Total 

294 

74.62 


100 

25.38 


394 

100.00 


TABLE  OF  TOP  BY  GEN 


TOP  GEN 


Frequency  1 
Percent  | 
Row  Pet  1 
Col  Pet  1 

11 

0 

1 

63  1 

65 

1 

15.99  1 

16.50 

i 

21.43  1 

22.11 

1 

63.00  1 

65.00 

1 

1 

37  1 

35 

1 

9.39  1 

8.88 

1 

37.00  1 

35.00 

1 

37.00  1 

35.00 

Total 

100 

100 

25.38 

25.38 

31 

41 

82 

— 1 — 

1 

- h 

84  1 

20.81 

1 

21.32  1 

27.89 

1 

28.57  1 

82.00 

1 

89.36  1 

18 

1 

10  1 

4.57 

1 

2.54  1 

18.00 

1 

10.00  1 

18.00 

1 

10.64  1 

100 

25.38 

“+— 

94 

23.86 

Total 

294 

74.62 


100 

25.38 


394 

100.00 
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Job  4  (Radio  Operator) 

TABLE  OF  TOP  BY  NEW 


TOP  NEW 


Frequency | 

Percent 

1 

Row  Pet 

! 

Col  Pet 

i 

11 

21 

3 

0 

1 

49  1 

67  1 

67 

1 

17.50  1 

23.93  1 

23.93 

1 

26.78  1 

36.61  1 

36.61 

1 

49.00  1 

67.00  1 

83.75 

1 

1 

51  1 

33  1 

13 

1 

18.21  i 

11.79  1 

4.64 

1 

52.58  1 

34.02  1 

13.40 

! 

51.00  1 

33.00  1 

16.25 

Total 

100 

100 

80 

35.71 

35.71 

28.57 

Frequency  Missing  =  9 


TOP 

Frequency 
Percent 
Row  Pet 
Col  Pet 


TABLE  OF  TOP  BY  SIG 
SIG 


II 


21 

— +- 


50  !  63  I  76 

17.30  I  21.80  I  26.30 

26.46  I  33.33  I  40.21 

50.00  I  63.00  I  85.39 


-+ 

1  I 

50  j 

37  1 

13 

1 

! 

17.30  1 

12.80  1 

4.50 

1 

1 

50.00  1 

37.00  1 

13.00 

1 

1 

50.00  1 

37.00  1 

14.61 

1 

Total 


100 

34.60 


100 

34.60 


89 

30.80 


TABLE 

OF 

TOP  BY 

GEN 

TOP 

GEN 

Frequency! 

Percent  | 

Row  Pet  t 

Col  Pet  1 

11 

21 

0  1 

45 

1 

70  [ 

74 

1 

15.57 

1 

24.22  1 

25.61 

1 

23.81 

1 

37.04  1 

39.15 

1 

45.00 

1 

70.00  1 

83.15 

1  1 

55 

1 

30  1 

15 

1 

19.03 

1 

10.38  1 

5.19 

1 

55.00 

1 

30.00  1 

15.00 

1 

55.00 

1 

30.00  1 

16.85 

Total 

100 

100 

89 

34.60 

34.60 

30.80 

Total 

183 

65.36 


97 

34.64 


280 

100.00 


Total 

189 

65.40 


100 

34.60 


289 

100.00 


Total 

189 

65.40 


100 

34.60 


289 

100.00 


125 


Job  5  (Vehicle  Mechanic) 


TABLE  OF  TOP  BY  NEW 


TOP 


NEW 


Frequency | 
Percent  i 
Row  Pet  1 
Col  Pet  i 

11 

2| 

31 

41 

5 

0 

1 

70 

1 

72 

1 

78 

1 

85  1 

73 

1 

14.64 

1 

15.06 

1 

16.32 

1 

17.78  1 

15.27 

1 

18.52 

1 

19.05 

1 

20,63 

1 

22.49  1 

19'.  31 

1 

70.00 

1 

72.00 

1 

78.00 

I 

85.00  1 

93:.  59 

1 

t 

30 

! 

28 

I 

22 

1 

15  1 

5 

1 

6.28 

1 

5.86 

1 

4.60 

1 

3.14  1 

r.05 

1 

30.00 

1 

28.00 

1 

22.00 

1 

15.00  1 

5^.00 

1 

30.00 

1 

28.00 

! 

22.00 

1 

15.00  1 

6.41 

- +— 

Total 

100 

100 

100 

100 

78 

20.92 

20.92 

20.92 

20.92 

16.32 

Total 

378 

79.08 


100 

20.92 


478 

100.00 


TABLE  OF  TOP  BY  SIG 


TOP 


SIG 


Frequency 
Percent 
Row  Pet 
Col  Pet 


II 


21 

— +- 


70 

14.64 

18.52 

70.00 


74 

15.48 

19.58 

74.00 


76 

15.90 

20.11 

76.00 


41 

— +- 


85 

17.78 

22.49 

85.00 


73 

15.27 

19.31 

93.59 


5! 

I 

I 


Total 

378 

79.08 


1  1 

1 

1  . 

1 

,  30  1 

6.28  1 
30.00  1 
30.00  1 

26  1 
5.44  1 
26.00  1 
26.00  1 

24  1 
5.02  I 
24.00  1 
24.00  1 

15  1 
3.14  1 
15.00  1 
15.00  1 

5  ! 
1.05  1 
5.00  1 
6.41  I 

100 

20.92 

Total 

100 

20.92 

100 

20.92 

100 

20.92 

100 

20.92 

78 

16.32 

478 

100.00 

TABLE  OF  TOP  BY  GEN 


TOP 


GEN 


Frequency! 
Percent  I 
Row  Pet  1 
Col  Pet  1 

11 

21 

31 

4i 

5 

0 

1 

68  1 

80 

1 

76 

-T  — 

1 

84  1 

70 

! 

14,23  1 

16,74 

} 

15.90 

1 

17.57  I 

14.64 

1 

17.99  1 

21.16 

1 

20.11 

1 

22.22  1 

18.52 

1 

68.00  1 

80.00 

1 

76.00 

1 

84.00  1 

89.74 

1 

1 

32  ) 

20 

! 

24 

1 

16  1 

8 

1 

6.69  1 

4.18 

1 

5.02 

! 

3.35  1 

1.67 

1 

32.00  I 

20.00 

1 

24.00 

1 

16.00  1 

8.00 

1 

-■+— 

32.00  1 

20.00 

1 

24.00 

1 

16.00  1 

10.26 

Total 

100 

100 

100 

100 

78 

20.92 

20.92 

20.92 

20.92 

16.32 

Total 

378 

79.08 


100 

20.92 


478 

100.00 
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Job  6  (Motor  Transport) 

TABLE  OF  TOP  BY  NEW 


TOP 


NEW 


Frequency 
Percent 
Row  Pet 
Col  Pet 


II 


57  I 
11.24  t 
14.00  I 
57.00  I 


21 


77 

15.19 

18.92 

77.00 


84 

16.57 

20.64 

84.00 


31 

! 


41 

— +- 


88 

17.36 

21.62 

88.:oo 


5 1  Total 


- + 

101  I 
19.92  I 
24.82  I 
94.39  I 
- + 


Total 


407 

80.28 


43  1 

23  1 

16  1 

12  1 

6  1 

100 

8.48  1 

4.54  1 

3.16  [ 

2.37  1 

1.18  1 

19.72 

43.00  1 

23.00  I 

16.00  ! 

12.(90  [ 

6.00  1 

43.00  1 

23.00  1 

16.00  1 

12.00  1 

5.61  1 

100 

100 

100 

100 

107 

507 

19.72 


19.72 


19.72 


19.72 


21.10 


100.00 


TOP 


SIG 


TABLE  OF  TOP  BY  SIG 


Frequency  I 
Percent  1 


Row 

Col 

Pet  I 
Pet  1 

11 

21 

31 

51 

Total 

0  1 

59  1 

76  1 

81  1 

89  1 

102  1 

407 

1 

11.64  1 

14.99  1 

15.98  1 

17.55  1 

20.12  1 

80.28 

1 

1 

- +- 

14.50  1 
59.00  1 
- +- 

18.67  1 
76.00  1 
- +- 

19.90  ! 
81.00  1 
- +- 

21.87  1 
89.00  1 
- +- 

25.06  1 
95.33  1 
- + 

I  ;  41  i 

I  8.09  ! 
I  41.00  i 
I  U.OO  I 


Total 


100 

19.72 


24  I 
4.73  I 
24.00  I 
24.00  I 

- + 

100 

19.72 


19 

3.75 

19.00 

19.00 


100 

19.72 


11  I 
2.17  I 
11.00  I 
11.00  I 

- - — +- 

100 

19.72 


5 

,99 

,00 

67 


107 

21.10 


100 

19.72 


507 

100.00 


TABLE  OF  TOP  BY  GEN 


TOP 


GEN 


Frequency  I 
Percent  | 
Row  Pet  I 
Col  Pet  I 


II 


21 


3| 


4| 


51  Total 


- 

- +- 

0  1  65  1 

70  1 

88  1 

82  ! 

102  1 

407 

I  12.82  1 

13.81  1 

17.36  1 

16.17  1 

20.12  1 

80.28 

1  15,97  1 

17.20  1 

21.62  ! 

20.15  1 

25.06  1 

1  65.00  1 

70.00  1 

88.00  1 

82.00  1 

95.33  1 
- + 

Total 


35  i 
6.90  ! 
35.00  ! 
35.00  1 

30  1 
5.92  I 
30.00  1 
30.00  1 

12  1 
2.37  1 
12.00  1 
12.00  1 

18  1 
3.55  1 
18.00  1 
18.00  1 

5  1 
0.99  1 
5.00  1 
4.67  1 

100 

19.72 

100 

100 

100 

100 

107 

507 

19.72' 

19.72 

19.72 

19.72 

21.10 

100.00 
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Job  7  (Administrative) 


TABLE  OF  TOP  BY  NEW 

TOP  NEW 

Frequency | 

Percent  | 

Row  Pet  I 

Col  Pet  1  II  „2i  3  1  4 1  Total 

- + - + - ^ - ^ - ^ 

01  55  1  77  I  81  I  114  I  327 

I  12.88  I  18.03  I  18.97  |  26.70  |  76.58 

I  16.82  I  23.55  |  24.77  t  34,86  1 

I  55.00  [  77.00  1  81.00  |  89.76  | 

- ^ - 4. - 4. - 4, - 4. 

II  45  I  23  I  19  t  13  I  100 

i  10.54  I  5.39  t  4.45  [  3.04  [  23.42 

I  45.00  1  23.00  I  19.00  |  13.00  | 

(  45.00  I  23.00  I  19.00  |  10.24  | 

- 4. - 4, - 4. - + - 4. 

Total  100  100  100  127  427 

23.42  23.42  23.42  29.74  100.00 


TABLE  OF  TOP  BY  SIG 

TOP  SIG 

Frequency | 

Percent  | 

Row  Pet  I 

Col  Pet  1  II  21  31  4!  Total 

- 4. - 4. - ^._4. - + - ^ 

0  i  56  I  72  I  85  I  114  i  327 

I  13.11  1  16.86  I  19.91  i  26.70  |  76.58 

I  17.13  I  22.02  I  25.99  I  34.86  | 

I  56.00  I  72.00  I  85.00  |  89.76  | 

- + - + - + - 4. - 4. 

1,1  44  I  28  I  15  I  13  I  100 

I  10.30  I  6.56  I  3.51  I  3.04  |  23.42 

I  44.00  I  28.00'  I  15.00  I  13.00  I 

I  44.00  I  28.00  I  15.00  I  10.24  | 

- + - + - -  + - 4- - 4. 

Total  100  100  100  127  427 

23.42  23.42  23.42  29.74  100.00 


TABLE  OF  TOP  BY  GEN 

TOP  GEN 

Frequency | 

Percent  I 
Row  Pet  1 

Col  Pet  I  11  2 1  31  4  1  Total 

- 4- - 4. - 4. - 4. - ^ 

0  !  60  I  76  I  79  i  112  I  327 

I  14.05  I  17.80  1  18.50  |  26.23  |  76.58 

I  18.35  I  23.24  |  24.16  |  34.25  ! 

I  60.00  j  76.00  I  79.00  |  88.19  | 

- + - 4. - 4. - 4. - 4. 

II  40  1  24  I  21  I  15  I  100 

I  9.37  I  5.62  I  4.92  |  3.51  |  23.42 

1  40.00  I  24.00  I  21.00  |  15.00  | 

I  40.00  1  24.00  I  21.00  i  11.81  | 

______  —  +  —  —  —  —  —  —  —  _  +  —  —  ___4._____ - (.________4. 

Total  100  100  100  127  427 

23.42  23.42  23.42  29.74  100.00 
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Job  8  (Medical) 


TABLE  OF  TOP  BY  NEW 

TOP  NEW 

Frequency | 

Percent  | 

Row  Pet  I 

Col  Pet  I  II  21  31  4  I  Total 

- + - + - - ^ - + 

01  59  I  70  [  80  1  83  I  292 

1  15.05  1  17.86  1  20.41  |  21.17  1  74.49 

i  20.21  .1  23.97  I  27.40  |  28.42  1 

I  59.00  ;|  70.00  |  80.00  |  90.22  | 

- + - + - + - + - + 

1  I  41  1  30  !  20  I  91  100 

!  10.46  ‘I  7,65  I  5.10  |  2.30  |  25.51 

I  41.00  1  30.00  !  20.00  |  9.00  1 

I  41.00  I  30.00  I  20.00  1  9.78  | 

- + - + - + - ^ - ^ 

Total  100  100  100  92  392 

25.51  25.51  25.51  23.47  100.00 


TABLE  OF  TOP  BY  SIG 

TOP  SIG 

Frequency  I 
Percent  | 

Row  Pet  1 

Col  Pet  1  1]  2  1  31  4  I  Total 

- + - + - - + - ^ 

0  I  60  I  69  I  81  1  82  I  292 

I  15.31  I  17.60  I  20.66  |  20.92  |  74.49 

I  20.55  I  23.63  |  27.74  |  28.08  | 

I  60.00  I  69.00  I  81.00  |  89.13  | 

- - + - + - + - + 

11  40  I  31  I  19  I  10  I  100 

1  10.20  \  7.91  I  4.85  |  2.55  |  25.51 

I  40.00  I  31.00  I  19.00  I  10.00  | 

I  40.00  )  31.00  I  19.00  I  10.87  | 

- + - + - ^ - ^ - ^ 

Total  100  100  100  92  392 

25.51  25.51  25.51  23.47  100.00 


Frequency  I 
Percent  I 

Row  Pet  I 

Col  Pet  I  II  21  31  4 1 

- 1 - 1- - i - 1 - ). 

01  59  I  71  I  79  I  83  I 

I  15.05  I  18.11  I  20.15  I  21.17  | 

I  20.21  I  24.32  I  27.05  |  28.42  I 

I  59.00  i  71.00  I  79.00  I  90.22  | 

- + - + - + - + - + 

II  41  I  29  I  21  I  91 

I  10.46  I  7.40  I  5.36  I  2.30  | 

I  41.00  I  29.00  i  21.00  I  9.00  I 

I  41.00  I  29.00  i  21.00  I  9.78  I 

- + - + - + - + - + 

Total  100  100  100  92 

25.51  25.51  25.51  23.47 


Total 

292 

74.49 


100 

25.51 


392 

100.00 
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Job  9  (Military  Police) 


TOP  NEW 


TABLE  OF  TOP  BY  NEW 


Frequency | 
Percent  | 
Row  Pet  I 


Col  Pet  I  -11  21  31  4 1  -51  61  Tot 

- + - ^ - ^ - ^ - ^ - ^ - ^ 

01  64  I  77  I  87  I  90  1  86  |  93  |  497 

1  10.72  I  12.90  I  14.57  |  15.08  |  14.41  [  15.58  1  83.25 

I  12.88  i  15.49  I  17.51  i  18.11  j  17.30  1  18.71  | 

I  64.00  I  77.00  [  87.00  |  90,00  |  .86.00  I  95,88;  | 

- + - + - + - + - + - -f - + 

II  36  I  23  1  13  I  10  i  14  I  4  I  100 

I  6.03  I  3.85  i  2.18  |  1,68  I  2.35  |  0.67' |  16.75 

1  36.00  I  23.00  I  13,00  |  10.00  1  '14.00  |  4.00^1 

I  36.00  I  23.00  I  13,00  I  10.00  |  14.00  1  4.12  | 

- + - + - + - + - + - + - + 

Total  100  100  100  100  100  97  597 

16.75  16.75  16.75  16.75  16.75  16.25^  100.00 


TABLE  OF  TOP  BY  SIG 


TOP  SIG 


Frequency  I 
Percent  | 

Row  Pet  I 

Col  Pet  I  'll  21  31  41  51  61 

- 1 - 4. - ^ - ^ - ^ - 1 - 

0  1  62  I  80  I  86  I  88  j  88  |  93  t 

!  10.39  I  13,40  !  14.41  1  14.74  1  14.74  |  15.58  | 

I  12.47  1  16.10  !  17.30  |  17.71  j  17.71  |  18.71  | 

I  62.00  I  80.00  I  86.00  |  88.00  }  88.00  |  95.88  | 

- 4. - ^ - - 4. - ^ - ^ - ^ - 


1  I  .  38  I  201  14  I  12  I  12  I  4.1 


1  6.37  I 

1  38.00  1 

{  38.00  1 

3.35  1 
20.00  [ 
20.00  1 

2.35  1 
14.00  1 
14.00  1 

2.01  I 
12.00  1 
12.00  1 

2.01  1 
12.00  f 
12.00  1 

0.67  1 
4.00  1 
4.12  1 

Total 

100 

100 

100 

100 

100 

97 

16.75  16.75  16.75  16.75  16,75  16.25 


TABLE  OF  TOP  BY  GEN 

TOP  GEN 

Frequency  I 
Percent  I 
Row  Pet  I 

Col  Pet  I  II  21  31  41  51  61  Tot 

- 4. - 4. - 4. - ^ - ^ - - 

0  I  65  I  76  I  87  I  88  I  89  |  92  |  497 

I  10.89  I  12.73  1  14.57  |  14.74  t  14.91  |  15.41  |  83.25 

I  13.08  1  15.29  I  17.51  |  17.71  |  17.91  |  18.51  | 

I  65.00  I  76.00  I  87.00  |  88.00  i  89.00  |  94.85  i 


- 4. - 4. - 4. - ^ - ^ - ^ 

35  I  24  I  13  I  12  I  11  I  51  100 


1  5.86  1 

1  35.00  1 

!  35.00  [ 

4.02  1 
24.00  1 
24.00  1 

2.18  1 
13.00  1 
13.00  1 

2.01  1 

12.00  1 

12.00  1 

1.84  1 
11.00  1 
11.00  1 

0.84  1 
5.00  1 
5.15  1 

16.75 

Total 

100 

100 

100 

100 

100 

97 

597 

16.75  16.75  16.75  16.75  16.75  16.25  100.00 
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Job  10  (Helicopter  Mechanic) 
TABLE  OF  TOP  ,BY  NEW 

TOP  NEW 


Frequencyl 
Percent  i 
Row  Pet  I 

Col  Pet  I  It  2  1  3i-  4  1  51  Total 

- + - + - + - ^ - ^ - ^ 

0  t  61  I  71  I  81  1:  87  I  39  i  339 

t  13.90  I  16.17  !  18.45  |  19.82  |  8.88  i  77.22 

I  17.99  i  20.94  I  23.89  f  25.66  |  ;  11.50  | 

1  61.00  1  71.00  !  81.00  87.00  |  -100.00  | 

- + - + - -f - + - + - + 

1  I  39  I  29  I  19  I  13  I  01  100 

I  8.88  I  6.61  I  4.33  ll  2.96  0.00  |  22.78 

I  39.00  I  29.00  I  19.00  I  13.00  |  0.00  I 

I  39.00  I  29.00  I  19.00  I  13.00  |  0.00  I 

1-  —  -.  - - - 

Total  100  100  100  100  39  439 

22.78  22.78  22.78  .  22.78  ,  8.88  100.00 


TABLE  OF  TOP  BY  SIG 

TOP  SIG 

Frequency ! 

Percent  ) 

Row  Pet  1 

Col  Pet  I  1|  2  1  31  4 1  51  Total 

- + - + - + - + - + - + 

01  61  I  70  I  83  I  86  I  39  |  339 

I  13.90  I  15.95  I  18.91  |  19.59  |  8.88  |  77.22 

1  17.99  I  20.65  I  24.48  |  25.37  |  11.50  j 

I  61.00  I  70.00  I  83.00  |  86.00  |  100.00  i 

- 1 - 1- - H - f. - 1- - ^ 

11  39  I  30  I  17  I  14  1  :  01  100 

1  ,  8.88  1  6.83  I  3.87  |  3.19  1  0.00  !  22.78 

I  39.00  !  30.00  I  17.00  |  14.00  1  0.00  | 

I  39.00  I  30.00  I  17.00  i  14.00  \  0.00  | 

- + - + - + - + - + - + 

Total  100  100  100  100  39  439 

22.78  22.78  22.78  22.78  8.88  100.00 


TABLE  OF  TOP  BY  GEN 

TOP  GEN 

Frequencyl 
Percent  1 
Row  Pet  I 

Col  Pet  I  1|  21  31  41  51  Total 

- + - + - + - + - + - + 

01  08  I  72  I  75  I  88  I  36  |  339 

I  15.49  I  16.40  I  17.08  |  20.05  |  8.20  |  77.22 

1  20.06  I  21.24  I  22.12  1  25.96  |  10.62  | 

I  68.00  I  72.00  I  75.00  I  88.00  i  92.31  I 

- + - + - ^ - - ^ - 

II  32  I  28  I  25  I  12  I  3  |  100 

i  7.29  I  6.38  i  5.69  I  2.73  |  0.68  i  22.78 

I  32.00  I  28.00  I  25.00  1  12.00  I  3.00  I 

I  32.00  I  28.00  i  25.00  |  12.00  I  7.69  | 

- + - 4. - ^ - ^ - ^ - 

Total  100  100  100  100  39  439 

•  22.78  22.78  22.78  22.78  8.88  100.00 


Job  11  (Automotive  Mechanic) 

TABLE  OF  TOP  BY  NEW 


TOP 


NEW 


Frequency 
Percent 
Row  Pet 
Col  Pet 


II 


21 


31 


41 


51 


61 


-+- 

- - - 

-+- 

-+- 

0 

1 

70 

1 

77 

1 

.  78 

1 

91 

1 

89 

i 

96 

1 

93 

I 

10.09 

1 

11,10 

1 

11:24 

1 

13.11 

I 

12.82 

1 

13.83 

1 

13,40 

! 

11,78 

1 

12.96 

1 

13-.13 

i 

15.32 

1 

14,98 

1 

16.16 

1 

15.66 

1 

70.00 

1 

77.00 

.1 

-0 

CD 

0 

0 

1 

91.00 

1 

89.00 

i 

96.00 

1 

98.94 

-+— 

1 

1 

30 

I 

23 

1 

22 

1 

9 

1 

11 

1 

4 

1 

1 

! 

4.32 

1 

3.31 

1 

3: 17 

1 

1.30 

1 

1.59 

1 

0.58 

1 

0.14 

I 

30.00 

1 

23.00 

'1 

22:00 

1 

9.00 

1 

11.00 

1 

4.00 

1 

1.00 

1 

-+- 

30,00 

1 

-+ ** 

23.00 

1 

22.00 

1 

9.00 

1 

11.00 

1 

4.00 

1 

1.06 

Total 

100 

100 

100 

100 

100 

•+— 

100 

94 

14.41 

14.41 

14.41 

14.41 

14.41 

14.41 

13.54 

71 

+ 


Total 

594 

85.59 


100 

14.41 


694 

100.00 


TOP 


SIG 


TABLE  OF  TOP  BY  SIG 


Frequency | 
Percent  | 
Row  Pet  1 
Col  Pet  I 
- +- 


11 


21 


I  70  I 
1  10.09  1 

I  11.78  I 
!  70.00  1 

-+ - +- 


77 

11.10 

12.96 

77.00 


31 

. 

51 

6i 

7| 

Total 

78  1 

92  1 

88  1 

96  1 

93  1 

594 

11.24  i 

13.26  1 

12.68  1 

13.83  1 

13.40  i 

85.59 

13.13  1 

15.49  1 

14.81  1 

16.16  1 

15.66  1 

78.00  1 

92.00  1 

88.00  1 

96.00  1 

98.94  1 

Total 


30  i 
4.32  1 
30.00  1 
30.00  1 

23  1 
3.31  1 
23.00  1 
23.00  I 

22  1 
3.17  1 
22.00  I 
22.00  1 

8  1 
1.15  f 
8.00  1 
8.00  1 

12  1 
1.73  1 
12.00  1 
12.00  I 

4  1 
0.58  1 
4.00  1 
4.00  1 

1  1 
0.14  1 
1.00  t 
1.06  1 

100 

14.41 

100 

14.41 

100 

14,41 

100 

14.41 

100 

14.41 

100 

14.41 

- h- 

100 

14.41 

94 

13.54 

694 

100.00 

TOP 


GEN 


TABLE  OF  TOP  BY  GEN 


Frequency 
Percent 
Row  Pet 
Col  Pet 


II 

— +- 


21 


31 


4  I 

— +- 


51 


Total 


0 

1 

1 

1 

1 

72 

10.37 

12.12 

72.00 

1 

f 

1 

i 

.4.— 

77  1 
11.10  1 
12.96  1 
77.00  1 

86  1 
12.39  1 
14.48  1 
86.00  1 

91 

13.11 

15.32 

91.00 

1 

1 

1 

1 

86  I 
12.39  1 
14.48  1 
86.00  1 

92 

13.26 

15.49 

92.00 

1 

1 

28 

1 

23  1 

14  1 

9 

1 

14  1 

8 

1 

4.03 

1 

3.31  1 

2.02  i 

1.30 

1 

2.02  1 

1.15 

! 

28.00 

1 

23.00  1 

14.00  1 

9.00 

1 

14.00  1 

8.00 

1 

28.00 

! 

23.00  1 

14.00  1 

9.00 

1 

14.00  1 

8.00 

100 

100 

100 

100 

100 

100 

14.41 

14.41 

14.41 

14.41 

14.41 

14.41 

61 


71 


90  1 
12.97  1 
15.15  I 
95.74  I 

- + 

4  I 
0.58  I 
4.00  I 
4.26  I 


94 

13.54 


Total 

594 

85.59 


100 

14.41 


694 

100.00 
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